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Reuven WEINTRAUB, et al. 



Applicant: 
Serial Number: 
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For: 
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Not Yet Assigned 

January 14, 1999 as PCT/IL99/00025 and Herewith as US National Stage 
ENHANCEMENT OF COMPRESSED IMAGES 
Not Yet Assigned 
Not Yet Assigned 



Honorable Commissioner of Patents and Trademarks 
Washington DC 20231 

PRELIMINARY AMENDMENT 

Sir: 

Further to the concurrent filing of the U.S. National stage of PCT/IL99/00025, kindly 
amend the application as follows prior to examination: 

IN THE SPECIFICATION 
Kindly add the following section, on page 1, immediately after the title: 

-RELATED APPLICATIONS 
The present application is a US national application of PCT/IL99/00025, filed on January 
14, 1999 and claims the benefit under 35 U.S.C. Section 119(e) of US Provisional Application 
No. 60/115,863 filed on January 13, 1999.- 

IN THE CLAIMS 

Kindly amend claims 6, 11, 18, 20, 24, 25, 27-30, 32, 34-36, 40, 50, 51, 53-63, by 
replacing the claims with the following amended claims, each of which is numbered with the 
number of the claim that it replaces. Please add new claims 64-67 which follow the amended 
claims. 

CLAIMS 

6. (Amended) A method according to claim 3 wherein determining an adjustment range 
comprises determining at least one adjustment limit and determining said adjustment range 
responsive to said adjustment limit. 

11. (Amended) A method according to claim 7 wherein said at least one adjustment limit for said 
at least one intermediate coefficient is determined responsive to the transformed quantizer having 
the same coordinates in the intermediate space as the at least one intermediate coefficient. 
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18. (Amended) A method according to claim 15 wherein said adjustment limit is equal to said 
difference multiplied by a fraction less than one. 

20. (Amended) A method according to claim 3 comprising testing said set of adjusted 
intermediate coefficients for consistency with said first set of coefficients in said transform space. 

24. (Amended) A method according to claim 20 wherein, if said set of intermediate coefficients is 
not consistent with said first set of coefficients, the value at least one adjusted intermediate 
coefficient of said set of adjusted intermediate coefficients is readjusted to generate a readjusted 
set of intermediate coefficients so that said set of readjusted intermediate coefficients is consistent 
with said first set of coefficients. 

25. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one intermediate coefficient responsive to coefficients 
in a neighborhood of said at least one intermediate coefficient. 

27. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one intermediate coefficient responsive to a ratio 
between a partial coefficient and a function of other partial coefficients. 

28. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to a trend in 
changes of values of partial coefficients as a function of changes in at least one coordinate of said 
intermediate space. 

29. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to an expected trend 
in changes of values of partial coefficients as a function of changes in at least one coordinate of 
said intermediate space. 

30. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to a predetermined 
template. 
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32. (Amended) A method according to claim 1 wherein adjusting the value of said at least one 
intermediate coefficient comprises adjusting the value of said at least one intermediate coefficient 
responsive to data from different compressed data that is related to said compressed data. 

34. (Amended) A method according to claim 32 wherein said different compressed data identify 
an edge in said recovered image. 

35. (Amended) A method according to claim 31 comprising smoothing said image on either side 
of said edge. 

36. (Amended) A method according to claim 1 comprising testing said set of adjusted 
intermediate coefficients for consistency with a first set of intermediate coefficients defined in an 
intermediate space different from said intermediate space in which said adjusted intermediate 
coefficients are defined. 

40. (Amended) A method according to claim 36 wherein if said set of adjusted intermediate 
coefficients is not consistent with said first set of intermediate coefficients, the value at least one 
adjusted intermediate coefficient of said set of adjusted intermediate coefficients is readjusted to 
generate a readjusted set of intermediate coefficients so that said set of readjusted intermediate 
coefficients is consistent with said first set of intermediate coefficients. 

50. (Amended) A method according to claim 46 wherein said plurality of said quantizers 
comprises all quantizers in said compressed data. 

51. (Amended) A method according to claim 46 wherein determining a value comprises 
determining a value responsive to image values of a different related image. 

53. (Amended) A method according to claim 1 comprising smoothing said recovered image. 

54. (Amended) A method according to claim 1 wherein said image is an image defined in a three 
dimensional image space. 

55. (Amended) A method according to claim 1 wherein said image is an image defined in a two 
dimensional space. 

3 
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56. (Amended) A method according to claim 1 wherein said compressed data is data generated 
using a unitary separable transform to transform an image in said image space into a set of values 
in said transform space. 

57. (Amended) A method according to claim 1 wherein said compressed data is generated using a 
JPEG compression method. 

58. (Amended) A method according to claim 1 wherein said compressed data is generated using 
an MPEG compression method. 

59. (Amended) A method according to claim 1 wherein said compressed data is generated using a 
PX64 compression method. 

60. (Amended) A method according to claim 1 wherein said compressed data is generated using 
an H261 compression method. 

61. (Amended) A method according to claim 1 wherein said compressed data is generated using 
an H263 compression method. 

62. (Amended) A method according to claim 1 wherein said compressed data is generated using 
an H323 compression method. 

63. (Amended) A method according to claim 1 wherein said compressed data is generated using 
an HDTV compression method. 

64. (New) A method according to claim 34 comprising smoothing said image on either side of 
said edge. 

65. (New) A method according to claim 48 wherein said plurality of said quantizers comprises all 
quantizers in said compressed data. 

66. (New) A method according claim 48 wherein determining a value comprises determining a 
value responsive to image values of a different related image. 
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67. (New) A method according to claim 65 wherein said image and said different related image 
are color component images of a multicolor color image. 

REMARKS 

The present application is a US National application of PCT/IL99/00025. The present 
amendments are based on the claims attached to the IPER and have been made to place the 
application in proper US form. Applicants note that the IPER established for PCT/IL99/00025 
indicates that claims 1-40 meet the requirements with respect to novelty, inventive step and 
industrial applicability of Articles 33(2), 33(3) and 33(4) of the PCT. 

The application contains 67 claims of which claims 64-67 are new claims. The added 
claims recite respectively the limitations of amended claims 35, 50, 51 and 52 and have been 
added to recite in single dependency form multiple dependencies recited in claims 35, 50, 51 and 
52 prior to their being amended. 

A clean copy of all the claims in the application is attached hereto. Also attached is a 
marked-up version of the claims. 

An action on the merits is respectfully awaited. 

Respectfully submitted, 
Reuven WEINTRAUB, et al. 

Paul Fenster 
Reg. No. 33,877 

July 11,2001 

William H. Dippert, Esq. 

c/o Cowan, Liebowitz and Latman, P.C. 

1 133 Avenue of the Americas 

New York, NY 10036-6799 

Tel: (212)790-9200 
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MARKED-UP VERSION OF AMENDED CLAIMS ONLY 

6. (Amended) A method according to any of claims 3 — 5 -claim 3 wherein determining an 
adjustment range comprises determining at least one adjustment limit and determining said 
5 adjustment range responsive to said adjustment limit. 

11. (Amended) A method according to any of claims 7 — l ^claim 7 wherein said at least one 
adjustment limit for said at least one intermediate coefficient is determined responsive to the 
transformed quantizer having the same coordinates in the intermediate space as the at least one 
10 intermediate coefficient. 

18. (Amended) A method according to any of claims 15 — 1- ^claim 15 wherein said adjustment 
limit is equal to said difference multiplied by a fraction less than one. 

15 20. (Amended) A method according to any of claims 3 — t9 -claim 3 comprising testing said set 
of adjusted intermediate coefficients for consistency with said first set of coefficients in said 
transform space. 

24. (Amended) A method according to any of claims 20 — 23 -elaim 20 wherein, if said set of 
20 intermediate coefficients is not consistent with said first set of coefficients, the value at least 
one adjusted intermediate coefficient of said set of adjusted intermediate coefficients is 
readjusted to generate a readjusted set of intermediate coefficients so that said set of readjusted 
intermediate coefficients is consistent with said first set of coefficients. 

25 25. (Amended) A method according to a ay of the preceding claims claim 1 w herein adjusting 
said at least one intermediate coefficient comprises adjusting said at least one intermediate 
coefficient responsive to coefficients in a neighborhood of said at least one intermediate 
coefficient. 

30 27. (Amended) A method according to claim 1 any of the preceding claimr, wherein adjusting 
said at least one intermediate coefficient comprises adjusting said at least one intermediate 
coefficient responsive to a ratio between a partial coefficient and a function of other partial 
coefficients. 



6 



079/02255 A01 



28. (Amended) A method according to claim 1 any of the preceding claims wherein adjusting 
said at least one intermediate coefficient comprises adjusting said at least one partial 
coefficient responsive to a trend in changes of values of partial coefficients as a function of 
changes in at least one coordinate of said intermediate space. 

5 

29. (Amended) A method according to claim 1 a ny o f the pr eceding claims wherein adjusting 
said at least one intermediate coefficient comprises adjusting said at least one partial 
coefficient responsive to an expected trend in changes of values of partial coefficients as a 
function of changes in at least one coordinate of said intermediate space. 

10 

30. (Amended) A method according to claim 1 a ny of the preceding claims wherein adjusting 
said at least one intermediate coefficient comprises adjusting said at least one partial 
coefficient responsive to a predetermined template. 

15 32. (Amended) A method according to daimj_ any of the preceding claims w herein adjusting 
the value of said at least one intermediate coefficient comprises adjusting the value of said at 
least one intermediate coefficient responsive to data from different compressed data that is 
related to said compressed data. 

20 34. (Amended) A method according to claim 32 or claim 33 wherein said different compressed 
data identify an edge in said recovered image. 

35. (Amended) A method according to claim 31 or claim 34 comprising smoothing said image 
on either side of said edge. 
25 36. (Amended) A method according to a ny of the previous claims claimj. comprising testing 
said set of adjusted intermediate coefficients for consistency with a first set of intermediate 
coefficients defined in an intermediate space different from said intermediate space in which 
said adjusted intermediate coefficients are defined. 

30 40. (Amended) A method according to any of claims 36 — 3 # claim 36 wherein if said set of 
adjusted intermediate coefficients is not consistent with said first set of intermediate 
coefficients, the value at least one adjusted intermediate coefficient of said set of adjusted 
intermediate coefficients is readjusted to generate a readjusted set of intermediate coefficients 
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so that said set of readjusted intermediate coefficients is consistent with said first set of 
intermediate coefficients. 



50. ( Amended) A method according to any of claims 46 — 4 9 claim 46 wherein said plurality of 
5 said quantizers comprises all quantizers in said compressed data. 

51. (Amended) A method according to any of claims 4 6 — 5 0 claim 46 wherein determining a 
value comprises determining a value responsive to image values of a different related image. 

10 53. (Amended) A method according to any of the preceding claims claim 1 comprising 
smoothing said recovered image. 



54. ( Amended) A method according to claim la 
is an image defined in a three dimensional image space. 



s-wherein said image 



55. (Amended) A method according to claim 1 1 
is an image defined in a two dimensional space. 



s-wherein said image 



56. (Amended) A method according to claim 1 any of the preceding claims wherein said 
20 compressed data is data generated using a unitary separable transform to transform an image in 

said image space into a set of values in said transform space. 

57. (Amended) A method according to claim 1 any of the precedi ng claims -wherein said 
compressed data is generated using a JPEG compression method. 

25 

58. (Amended) A method according to claim 1 any of claims 1-56 wherein said compressed 
data is generated using an MPEG compression method. 

59. (Amended) A method according to claim 1 any of claims 1 — S4-wherein said compressed 
30 data is generated using a PX64 compression method. 

60. (Amended) A method according to claim 1 any of cla i ms 1 - 56 wherein said compressed 
data is generated using an H261 compression method. 
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61. (Amended) A method according to claim 1 any of claims 1 - 56 wherein said compressed 
data is generated using an H263 compression method. 

62. (Amended) A method according to claim 1 any of claims 1- 56 w herein said compressed 
5 data is generated using an H323 compression method. 

63. (Amended) A method according to claim 1 any o f cla ims 1 - 56 wherein said compressed 
data is generated using an HDTV compression method. 

10 64. (New) A method according to claim 34 comprising smoothing said image on either side of 
said edge. 

65. (New) A method according to claim 48 wherein said plurality of said quantizers comprises 
all quantizers in said compressed data. 

15 

66. (New) A method according claim 48 wherein determining a value comprises determining a 
value responsive to image values of a different related image. 

67. (New) A method according to claim 65 wherein said image and said different related image 
20 are color component images of a multicolor color image. 
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CLEAN VERSION OF ALL THE CLAIMS 

1. A method for recovering an image defined as a function of image coordinates of an 
image space from compressed data that is a function of transform coordinates in a transform 
space comprising: 

a) transforming said compressed data using at least one partial transform that 
transforms at least one but not all of the transform coordinates into a corresponding image 
coordinate to generate a set of intermediate coefficients which are functions of at least one 
coordinate from the image space and at least one coordinate from the transform space in a 
space intermediate between said transform space and said image space said intermediate space 
being defined by at least one coordinate from the image space and at least one coordinate from 
the transform space; 

b) adjusting the value of at least one intermediate coefficient and using said adjusted 
value to generate a set of adjusted intermediate coefficients; and 

c) recovering said image by transforming said set of adjusted intermediate coefficients 
to said image space with at least one additional partial transform. 

2. A method according to claim 1 wherein adjusting the value of at least one partial 
coefficient comprises: 

a) determining an adjustment range; 

b) determining a value in said adjustment range; and 

c) setting the value of said intermediate coefficient equal to said determined value. 

3. A method according to claim 2 wherein said compressed data comprises information 
for determining a set of quantized values and a quantizer for each quantized value as functions 
of coordinates of said transform space, and wherein transforming said data comprises 
determining a first set of coefficients in said transform space using said quantizers and said 
quantized values and transforming said first set of coefficients with said at least one partial 
transform. 

4. A method according to claim 3 wherein determining said first set of coefficients 
comprises determining the product of each quantized value with its quantizer and determining 
said first set of coefficients as a set comprising all said products. 
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5. A method according to claim 3 wherein determining said first set of coefficients 
comprises determining the product of each quantized value with its quantizer, adjusting the 
value of at least one of said products and determining said first set of coefficients as a set 

5 comprising all said products. 

6. (Amended) A method according to claim 3 wherein determining an adjustment range 
comprises determining at least one adjustment limit and determining said adjustment range 
responsive to said adjustment limit. 

10 

7. A method according to claim 6 wherein determining at least one adjustment limit 
comprises transforming said set of quantizers with said at least one transform to generate a set 
of transformed quantizers in said intermediate space and determining an adjustment limit for at 
least one intermediate coefficient responsive to at least one transformed quantizer of said set of 

1 5 transformed quantizers . 

8. A method according to claim 7 comprising adjusting the value of at least one quantizer 
responsive to values of said first coefficients prior to transforming said set of quantizers. 

20 9. A method according to claim 8 wherein adjusting the value of at least one quantizer 
comprises determining whether all of said first set of coefficients are zero for a value of a 
single coordinate of said transform space greater than a certain value and constant values for 
all other transform space coordinates and if so, setting the values of the quantizers 
corresponding to said zero coefficients equal to a same value. 

25 

10. A method according to claim 9 wherein said same value is equal to the maximum of 
said coefficients for all values of said single coordinate and said constant values for all other 
coordinates. 

30 11. (Amended) A method according to claim 7 wherein said at least one adjustment limit for 
said at least one intermediate coefficient is determined responsive to the transformed quantizer 
having the same coordinates in the intermediate space as the at least one intermediate 
coefficient. 
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12. A method according to claim 11 wherein said adjustment limit is equal to said 
corresponding transformed quantizer multiplied by a fraction less than one. 

13. A method according to claim 12 wherein said fraction is equal to 0.5. 

14. A method according to claim 6 wherein determining at least one adjustment limit 
comprises determining an adjustment limit for at least one intermediate coefficient responsive 
to the value of at least one of said intermediate coefficients. 

15. A method according to claim 14 wherein determining said at least one adjustment limit 
comprises determining the difference between a maximum and minimum intermediate 
coefficient in a plurality of intermediate coefficients in said set of intermediate coefficients and 
determining said adjustment limit responsive to said difference. 

16. A method according to claim 15 wherein said plurality of intermediate coefficients 
comprises intermediate coefficients in a neighborhood of said at least one intermediate 
coefficient. 

17. A method according to claim 15 wherein all but one of the coordinates of any two of 
the plurality of intermediate coefficients are the same and wherein the plurality of 
intermediate coefficients includes the at least one intermediate coefficient. 

18. (Amended) A method according to claim 15 wherein said adjustment limit is equal to said 
difference multiplied by a fraction less than one. 

19. A method according to claim 16 wherein said fraction is equal to 0.5. 

20. (Amended) A method according to claim 3 comprising testing said set of adjusted 
intermediate coefficients for consistency with said first set of coefficients in said transform 
space. 

21. A method according to claim 20 wherein testing said set of adjusted intermediate 
coefficients comprises transforming said set of adjusted intermediate coefficients into a second 
set of coefficients in said transform space and comparing coefficients of said first set with 

12 



079/02255 A01 



coefficients of said second set to determine if said set of adjusted coefficients is consistent with 
said first set of coefficients. 

22. A method according to claim 21 wherein comparing coefficients comprises 
5 determining how close the value of at least one coefficient of said first set is to the value of at 

least one coefficient of said second set. 

23. A method according to claim 22 wherein determining how close the value of at least 
one coefficient of said first set is to the value of at least one coefficient of said second set 

10 comprises using a metric. 

24. (Amended) A method according to claim 20 wherein, if said set of intermediate 
coefficients is not consistent with said first set of coefficients, the value at least one adjusted 
intermediate coefficient of said set of adjusted intermediate coefficients is readjusted to 

15 generate a readjusted set of intermediate coefficients so that said set of readjusted intermediate 
coefficients is consistent with said first set of coefficients. 

25. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one intermediate coefficient responsive to 

20 coefficients in a neighborhood of said at least one intermediate coefficient. 

26. A method according to claim 25 wherein said neighborhood comprises a plurality of 
intermediate coefficients, including said at least one intermediate coefficient, for which any 
two intermediate coefficients have same values for all the same coordinates except one. 

25 

27. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one intermediate coefficient responsive to a ratio 
between a partial coefficient and a function of other partial coefficients. 

30 28. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to a trend in 
changes of values of partial coefficients as a function of changes in at least one coordinate of 
said intermediate space. 
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29. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to an expected 
trend in changes of values of partial coefficients as a function of changes in at least one 
coordinate of said intermediate space. 

5 

30. (Amended) A method according to claim 1 wherein adjusting said at least one intermediate 
coefficient comprises adjusting said at least one partial coefficient responsive to a 
predetermined template. 

10 31. A method according to claim 30 wherein said template identifies an edge in said 
recovered image and comprising smoothing said recovered image on either side of said 
identified edge. 

32. (Amended) A method according to claim 1 wherein adjusting the value of said at least one 
15 intermediate coefficient comprises adjusting the value of said at least one intermediate 

coefficient responsive to data from different compressed data that is related to said compressed 
data. 

33. A method according to claim 32 wherein said image in said image space comprises 
20 values for a first color component of a color image in said image space and said different 

compressed data comprises data for generating a second color component image of said color 
image. 

34. (Amended) A method according to claim 32 wherein said different compressed data 
25 identify an edge in said recovered image. 

35. (Amended) A method according to claim 31 comprising smoothing said image on either 
side of said edge. 

30 36. (Amended) A method according to claim 1 comprising testing said set of adjusted 
intermediate coefficients for consistency with a first set of intermediate coefficients defined in 
an intermediate space different from said intermediate space in which said adjusted 
intermediate coefficients are defined. 
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37. A method according to claim 36 wherein testing said set of adjusted intermediate 
coefficients comprises transforming said set of adjusted intermediate coefficients into a second 
set of intermediate coefficients in said different intermediate space and comparing intermediate 
coefficients of said first set with intermediate coefficients of said second set to determine if 

5 said set of adjusted coefficients is consistent with said first set of intermediate coefficients. 

38. A method according to claim 37 wherein comparing coefficients comprises 
determining how close the value of at least one intermediate coefficient of said first set is to 
the value of at least one intermediate coefficient of said second set. 

10 

39. A method according to claim 38 wherein determining how close the value of at least 
one intermediate coefficient of said first set is to the value of at least one intermediate 
coefficient of said second set comprises using a metric. 

15 40. (Amended) A method according to claim 36 wherein if said set of adjusted intermediate 
coefficients is not consistent with said first set of intermediate coefficients, the value at least 
one adjusted intermediate coefficient of said set of adjusted intermediate coefficients is 
readjusted to generate a readjusted set of intermediate coefficients so that said set of readjusted 
intermediate coefficients is consistent with said first set of intermediate coefficients. 

20 

41. A method for recovering an image defined as a function of image coordinates in an 
image space from compressed data that is a function of transform coordinates in a transform 
space the method comprising: 

a) using said data to generate a first set of recovered transform coefficients in said 
25 transform space; 

b) determining an adjustment range for at least one recovered transform coefficient; 

c) replacing said at least one recovered transform coefficient by a value in said 
adjustment range responsive to recovered transform coefficients in a neighborhood comprising 
less than all of said recovered transform coefficients of said at least one recovered transform 

30 coefficient, to generate an adjusted set of recovered transform coefficients; and 

d) recovering said image by transforming said adjusted set of recovered transform 
coefficients to said image space. 
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42. A method according to claim 41 wherein said neighborhood comprises a plurality of 
recovered transform coefficients including said at least one recovered coefficient, for which 
any two recovered coefficients have same values for all the same coordinates except one. 

5 43. A method for recovering an image defined in an image space from compressed data 
that comprises quantized values and a quantizer for each quantized value that are functions of 
transform coordinates in a transform space the method comprising: 

a) determining a set of recovered transform coefficients in which each recovered 
transform coefficient is a product of a quantized value with its quantizer; 
10 b) determining an adjustment range having an upper and lower limit for at least one 

recovered transform coefficient of said first set of recovered coefficients, wherein the 
difference between said upper and lower limit is less than the magnitude of said quantizer used 
to determine said at least one recovered coefficient ; 

c) replacing said at least one recovered transform coefficient by a value in said 
15 adjustment range to generate an adjusted set of recovered transform coefficients; and 

d) recovering said image by transforming said adjusted set of recovered transform 
coefficients to said image space. 

44. A method for recovering an image defined in an image space from a first set of 
20 compressed data that comprises quantized values and a quantizer for each quantized value that 

are functions of transform coordinates in a transform space the method comprising: 

a) determining a first set of recovered transform coefficients in which each recovered 
transform coefficient is a product of a quantized value with its quantizer; 

b) determining an adjustment range for at least one recovered coefficient; 

25 c) replacing said at least one transform coefficient with a value in said adjustment range 

responsive to recovered transform coefficients generated using a second set of related 
compressed data to generate an adjusted set of recovered transform coefficients; and 

d) recovering said image by transforming said adjusted set of recovered transform 
coefficients to said image space. 

30 

45. A method according to claim 44 wherein said first and second sets of compressed data 
are generated from different color component images of a color image. 
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46. A method for adjusting an image having a plurality of image values defined in an 
image space, said image values being generated from compressed data comprising quantized 
values and a quantizer for each quantized value that are functions of transform coordinates in a 
transform space the method comprising: 

5 determining an adjustment range for at least one image value of said plurality of image 

values responsive to an average of a plurality said quantizers; 
determining a value in said adjustment range; 

adjusting said image by replacing said at least one image value with said determined 

value. 

10 

47. A method for adjusting an image according to claim 46 wherein determining an 
adjustment range comprises determining an adjustment range responsive to a fraction less than 
one of the average quantizer. 

15 48. A method for adjusting an image having a plurality of image values defined in an 
image space, said image values being generated from compressed data comprising quantized 
values and a quantizer for each quantized value that are functions of transform coordinates in a 
transform space the method comprising: 

determining an adjustment range for at least one image value of said plurality of image 
20 values responsive to a maximum or a minimum quantizer of a plurality of said quantizers; 
determining a value in said adjustment range; 

adjusting said image by replacing said at least one image value with said determined 

value. 

25 49. A method for adjusting an image according to claim 48 wherein determining an 
adjustment range comprises determining an adjustment range responsive to a fraction less than 
one of the maximum or minimum quantizer. 

50. (Amended) A method according to claim 46 wherein said plurality of said quantizers 
30 comprises all quantizers in said compressed data. 

51. (Amended) A method according to claim 46 wherein determining a value comprises 
determining a value responsive to image values of a different related image. 
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52. A method according to claim 51 wherein said image and said different related image 
are color component images of a multicolor color image. 

53. (Amended) A method according to claim 1 comprising smoothing said recovered image. 

5 

54. (Amended) A method according to claim lwherein said image is an image defined in a 
three dimensional image space. 

55. (Amended) A method according to claim 1 wherein said image is an image defined in a 
10 two dimensional space. 

56. (Amended) A method according to claim 1 wherein said compressed data is data generated 
using a unitary separable transform to transform an image in said image space into a set of 
values in said transform space. 

15 

57. (Amended) A method according to claim 1 wherein said compressed data is generated 
using a JPEG compression method. 

58. (Amended) A method according to claim 1 wherein said compressed data is generated 
20 using an MPEG compression method. 

59. (Amended) A method according to claim 1 wherein said compressed data is generated 
using a PX64 compression method. 

25 60. (Amended) A method according to claim 1 wherein said compressed data is generated 
using an H261 compression method. 

61. (Amended) A method according to claim 1 wherein said compressed data is generated 
using an H263 compression method. 

30 

62. (Amended) A method according to claim 1 wherein said compressed data is generated 
using an H323 compression method. 
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63. (Amended) A method according to claim 1 wherein said compressed data is generated 
using an HDTV compression method. 

64. (New) A method according to claim 34 comprising smoothing said image on either side of 
5 said edge. 

65. (New) A method according to claim 48 wherein said plurality of said quantizers comprises 
all quantizers in said compressed data. 

10 66. (New) A method according claim 48 wherein determining a value comprises determining a 
value responsive to image values of a different related image. 

67. (New) A method according to claim 65 wherein said image and said different related image 
are color component images of a multicolor color image. 
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ENHANCEMENT OF COMPRESSED IMAGES 



FIELD OF THE INVENTION 



The present invention relates to processing compressed data and in particular to 
processing compressed data representing images. 



Still and moving images generally contain large amounts of information. For example, 
an RGB color image comprising 2000 x 2000 pixels requires 12 megabytes of data if the 
intensity of each RGB color component of each pixel is coded with 8 bits. Transmission and 
storage of this amount of data even for single images is generally impractical for most 
purposes. For example, if images of this size are to be transmitted at typical moving picture 
rates of thirty frames per second, then 360 megabytes of image data must be transmitted every 
second. This is clearly a very large rate of data transmission. Therefore, data representing an 
image is generally "compressed". Compression significantly reduces the amount of data 
representing the image in comparison to the amount of data required to represent the image by 
coding RGB intensities (or intensities of a different set of color components used for defining 
color) for each pixel of an image. Hereinafter, the intensity of a color component used to 
define color of a pixel in an image is referred to as a "pixel value". 

Different techniques exist for compressing data and in particular for compressing data 
representing an image. In many of these techniques the image is first partitioned into a plurality 
of contiguous, non-overlapping, sub-images or tiles wherein each sub-image extends over a 
portion of the area of the full image. Variation of image intensity and color over the area of a 
sub-image is generally limited. For each color component used to define color in the image, 
pixel values of each sub-image are transformed into a set of values, hereinafter referred to as 
"transform coefficients", in a transform space. Generally, the pixel values are transformed by 
operating on the pixel values with a unitary separable transform. The pixel values are 
recoverable from the transform coefficients by operating on the transform coefficients with an 
inverse of the transform. Among well known separable unitary transforms used in processing 
image data are the Fourier, cosine, sine and Hadamard transforms. 

Generally, the number of transform coefficients needed to effectively recover the pixel 
values is considerably less than the number of pixel values. The amount of data needed to code 
the transform coefficients is therefore generally significantly less than the amount of data 
needed to code the pixel values. Representing the image using the transform coefficients 
therefore requires considerably less data than representing the image by coding color 
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component intensities for each pixel. The transform thus succeeds in compressing the image 
data. 

Usually, the transform coefficients are quantized by dividing each transform coefficient 
by a "quantizer". Any remainder resulting from the division is rounded up or down and the 
5 transform coefficient is replaced by the quotient. Different transform coefficients are generally 
quantized with quantizers of different magnitudes. Often the magnitude of a quantizer is 
determined as a function of an error margin for the value of the transform coefficient that the 
quantizer is used to quantize, which error margin is estimated from known error margins of the 
pixel values. 

1 0 Transform space data representing the image after quantizing the transform coefficients 

comprises the quantized transform coefficients and the quantizer, or a way of determining the 
quantizer, for each quantized transform coefficient. When recovering the image, i.e. when 
recovering the pixel values, a transform coefficient, hereinafter referred to as a "recovered 
transform coefficient", is recovered from each quantized transform coefficient by multiplying 

1 5 the quantized transform coefficient by its quantizer. The recovered transform coefficients are 
operated on by an inverse of the transform used to generate the transform coefficients, to 
determine the pixel values. 

Quantization increases the compression of the image data beyond that achieved with the 
transform. Quantization reduces the range of different numbers representing the transform 

20 coefficients, which reduces the number of bits required to code the coefficients. In addition 
some of the coefficients (those that are less than one half their quantizer) are quantized to zero, 
which reduces the number of coefficients used to recover the original pixel values. Increasing 
quantizer magnitudes decreases the possible different values of quantized transform 
coefficients and increases the number of transform coefficients that are quantized to zero. 

25 Increasing quantizer magnitudes therefore generally increases the extent to which image data is 
compressed. 

However, unlike compression resulting from a non-quantized unitary transform, 
compression from quantization is "lossy". Information contained in the remainders discarded in 
the quantization process is lost and the larger the quantizer magnitudes the more lossy is the 
30 compression. The recovered transform coefficients are different from the transform coefficients 
by a quantization error. As a result, pixel values generated using quantized transform 
coefficients differ from the original pixel values. 
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The steps in the industry standard JPEG system for compressing color and gray tone 
images illustrates a typical data compression procedure for a color image. In JPEG a color 
image is generally partitioned into square tiles of 8 x 8 pixels. Color of the image is usually 
defined using YUV color components, so that each pixel has Y, U and V pixel values. For each 
tile, for each of the YUV color components, the pixel values of the color component are 
transformed into a set of transform coefficients using a discrete cosine transform. The 
transform coefficients are then quantized. Usually the transform coefficients associated with 
higher spatial frequencies are quantized with larger quantizers. 

To illustrate the process, let Y(x,y) represent luminance pixel values in a tile, where x 
and y are integer coordinates that locate a pixel in the tile as being in the x-th row and y-th 
column of the tile. The transform coefficients are functions of two integer transform space 
coordinates, "u" and "v" that are conjugate coordinates of x and y respectively. 

If C(u,v) represents the transform coefficients, then C(u,v) 
7 7 

= Z XDCT(u,v,x,y)L(x,y) where DCT(u,v,x,y) symbolically represents the discrete 
y=0x=0 

cosine transform. Writing the discrete cosine transform explicitly gives: 
7 7 

C(u,v)=A(u)A(v) Z ZL(x,y)cos((2x + l)uW16)cos((2y+ I)vW16), where A(w) = 1/ V2 
y=0x = 0 

for w = 0 and 1 otherwise. Each C(u,v) is then quantized with a quantizer, Q(u,v), yielding a 
quantized transform coefficient QC{u,v) = INT{[C(u,v) ± Q(u,v)/2]/Q(u,v)}, where the + is 
used if C(u,v) > 0, the minus is used if C(u,v) < 0 and INT represents rounding the result to the 
nearest integer. 

To recover pixel values for luminance from the QC(u,v), recovered transform 
coefficients, RC(u,v), are calculated where RC(u,v) = QC(u,v)Q(u,v). The recovered transform 
coefficients are then transformed back with the inverse discrete cosine transform represented 
by IDCT(x,y,u,v), so that recovered luminance values RY(x,y) may be written: 
7 7 

RY(x,y)= 2 ZrDCT(x,y,u,v)RC(u,v). Explicitly, 
u=0v=0 

7 7 

RY(x,y)=A(u)A(v) 2 SRC(u,v)cos((2x + l)u^/16)cos((2y + l)v^r/16) . The recovered 
u=0v=0 

value for luminance RY(x,y) is generally not equal to Y(x,y) because the quantization process 
has resulted in a loss of information so that generally RC(u,v) * C(u,v). 
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In many instances, information lost in quantizing transform coefficients does not affect 
the quality of an image recovered from the transform coefficients to an extent that renders the 
image unusable for the purpose for which it was intended. However, in many instances the loss 
of information results in objectionable degradation of the recovered image. Often, unwanted 
artifacts are generated in the recovered image and often the image is degraded unacceptably. 
For example, a set of compressed transform coefficients might be useable to provide a good 
"thumbnail" image of a scene but as a result of information loss the transform coefficients may 
be totally inadequate to provide an enlarged image of the scene. 

Techniques exist for adjusting images recovered from compressed data so as to reduce 
effects on the quality of the recovered image that result from information loss and error in the 
compressed data. Some techniques address specific types of defects or artifacts, for example 
"blocking artifacts", that are generated in images recovered from compressed data that has lost 
information. Other techniques apply algorithms to reduce pixel to pixel discontinuities or to 
improve image sharpness by edge enhancement. Most such techniques have limitations. For 
example, edge-enhancing algorithms typically increase noise in a recovered image and 
smoothing algorithms tend to blur an image. There is a need for new techniques for recovering 
images from compressed data that compensate for information loss or errors in the compressed 
data. 

Whereas the above discussion has focused on data representing two dimensional images 
comprising pixels and associated pixel values of color components or gray tones, it should be 
realized that the discussion also applies to three dimensional data sets comprising voxels and 
corresponding "voxel values". Quite generally, the present discussion is germane to n- 
dimensional images. An n-dimensional image is defined as a set of values, hereinafter referred 
to as "image values", that are dependent on n independent coordinates, which coordinates 
define an n-dimensional image space. Furthermore it should be noted that the definition of an 
n-dimensional image is quite general and an n-dimensional image is not restricted to a video 
image, it means, among other things, a sound image as well. For example a three dimensional 
sound image might be the notes in a sound file that are a function of time and two stereo 
channels. 

Compression of data representing an n-dimensional image can be similar to 
compression of a two dimensional video image comprising pixel values as illustrated with the 
JPEG example. Image values of the n-dimensional image are transformed preferably by a 
separable unitary transform into a set of transform coefficients dependent upon transform space 
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coordinates of an n-dimensional transform space. The transform coefficients are then quantised 
using appropriate quantizers. Each of the transform space coordinates is conjugate to a different 
one of the coordinates of the image space. An image is recovered from transform coefficients 
by dequantizing the quantized transform coefficients to determine recovered transform 
coefficients and operating on the recovered transform coefficients with an inverse of the unitary 
transform. As in the case of two dimensional images the recovered image is degraded by loss 
of information, or error in the recovered transform coefficients and techniques are needed to 
compensate for the information loss. 

SUMMARY OF THE INVENTION 

An aspect of preferred embodiments of the present invention relates to providing a 
method for adjusting compressed data comprising quantized transform coefficients generated 
by transforming image data, preferably, with a separable unitary transform. The method is 
useable to moderate deleterious effects on an image recovered from the compressed data that 
result from information loss and/or error in the compressed data. 

Aspects of some preferred embodiments of the present invention relate to processing 
compressed data that represents an n-dimensional image. 

In accordance with aspects of some preferred embodiments of the present invention the 
method relates to processing compressed data generated from two and three-dimensional 
images. 

Aspects of some preferred embodiments of the present invention relate to processing 
compressed data generated by compressing image data using the JPEG system and derivatives 
of the JPEG system such as MPEG and related and similar compression schemes such as H261, 
H263, H323, PX64 and HDTV. 

Preferably, the transform used to compress the data is separable so that it can be 
decomposed into a set of partial transforms that operate sequentially to transform the image 
values into the transform coefficients. A first partial transform operates on the image data and 
generates a set of first partial transform coefficients. A second partial unitary transform 
operates on the set of first partial transform coefficients to generate a set of second partial 
transform coefficients. The procedure continues with each partial transform operating on partial 
transform coefficients generated by the preceding partial transform. The results of the last 
partial transform are the transform coefficients. In the process of transforming the data each 
partial transform converts at least one coordinate of the image space into its conjugate 
transform coordinate in the transform space. The transform can thus be decomposed into, at 
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most, n partial transforms in which each partial transform converts one different image space 
coordinate into its transform space coordinate. 

The inverse transform can preferably be decomposed into corresponding partial inverse 
transforms in which each partial inverse transform is an inverse of a corresponding partial 
transform. In preferred embodiments of the present invention, the partial inverse transforms 
operate sequentially to "roll back" the results of the partial transforms, each partial inverse 
transform rolling back the results of a corresponding partial transform. If a partial transform 
generates a set of third partial transform coefficients from a set of second partial transform 
coefficients then its corresponding partial inverse transform generates the set of second 
transform coefficients from the set of third transform coefficients. • 

Each partial transform coefficient, hereinafter referred to as a "partial coefficient", is a 
function of at least one transform space coordinate and at least one image space coordinate. 
Partial coefficients generated by a same sequence of partial transforms and/or partial inverse 
transforms are functions of the same transform space coordinates and the same image space 
coordinates and are said to belong to a same set of partial coefficients. While in preferred 
embodiments of the present invention both transforms have the above characteristics, in other 
preferred embodiments of the present invention only the inverse transform is required to be 
separable in that the total inverse transform can be achieved by operation on the compressed 
data by a sequence of partial inverse transforms. 

In accordance with a preferred embodiment of the present invention the compressed 
data is adjusted by adjusting at least one partial coefficient to provide at least one adjusted 
partial coefficient. The at least one adjusted partial coefficient is used in place of the at least 
one partial coefficient in recovering an image from the transform coefficients. 

For at least one partial coefficient an adjustment limit is preferably determined that is 
used to set an upper and lower bound for the amount by which the partial coefficient may be 
increased or decreased by an adjustment. The range of values determined by the upper and 
lower bound are hereinafter referred to as an "adjustment range". The adjustment comprises 
using an adjustment algorithm to assign a value to the at least one partial coefficient that lies in 
the adjustment range. Using the algorithm does not necessarily result in a change in the value 
of the at least one partial coefficient. In some cases the result of using the algorithm is to leave 
the value of the at least one partial coefficient unchanged. 

According to an adjustment algorithm used in some preferred embodiments of the 
present invention, the value of the at least one partial coefficient is adjusted so as to moderate 
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at least one apparent anomaly in values of partial coefficients belonging to the set of partial 
coefficients to which the partial coefficient belongs. 

According to an adjustment algorithm used in some preferred embodiments of the 
present invention, the value of the at least one partial coefficient is adjusted so as to conform to 
ratios between other partial coefficients in the same set of partial coefficients. 

According to an adjustment algorithm used in some preferred embodiments of the 
present invention, the value of the at least one partial coefficient is adjusted to conform to a 
trend in changes in values of partial coefficients as a function of at least one of the transform 
space coordinates or one of the image space coordinates. 

According to an adjustment algorithm used in some preferred embodiments of the 
present invention, the value of the at least one partial coefficient is adjusted to conform to a 
predetermined pattern relating values of partial coefficients. 

For compressed data representing color images, such as, for example, data compressed 
using JPEG, MPEG and derivative compression systems, in some preferred embodiments of 
the present invention, the value of at least one partial coefficient for one color component is 
adjusted responsive to at least one value of a transform coefficient or a partial coefficient for a 
different color component. For example a partial coefficient for a chrominance component U or 
V may be adjusted using partial coefficients for luminance Y. 

Quite generally, if first and second sets of compressed data are from related images then 
recovered coefficients and/or partial coefficients generated from the first set of compressed data 
may be adjusted, in accordance with a preferred embodiment of the present invention, 
responsive to recovered coefficients and or partial coefficients of the second set of compressed 
data. 

In some preferred embodiments of the present invention, if the value of at least one 
partial coefficient is changed in a set of partial coefficients, the at least one changed value is 
checked for consistency with the value of at least one recovered transform coefficient. In some 
preferred embodiments of the present invention, the check is performed by transforming the 
"adjusted" set of partial coefficients using the partial transform that transforms the set of partial 
coefficients into the set of transform coefficients (i.e. the inverse of the partial transform that 
transforms the set of transform coefficients into the set of partial coefficients). At least one 
value of the transformed set of adjusted partial coefficients is then compared with at least one 
transform coefficient using an appropriate metric to determine if the set of adjusted partial 
coefficients are consistent with the transform coefficients. 
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In some preferred embodiments of the present invention, if the value of at least one 
partial coefficient is changed in a first set of partial coefficients, the at least one changed value 
is checked for consistency with the value of at least one partial coefficient in a second set of 
partial coefficients. In some preferred embodiments of the present invention, the check is 
5 performed by transforming the adjusted first set of partial coefficients using the partial 
transform or partial inverse transform that transforms the first set of partial coefficients into the 
second set of partial coefficients. The transformed set of adjusted first partial coefficients is 
then compared with the second set of partial coefficients using an appropriate metric to 
determine if the first and second sets of partial coefficients are consistent. 
10 In some preferred embodiments of the present invention an adjustment range for a 

partial coefficient is determined responsive to values of other partial coefficients in the same 
set of partial coefficients. 

In some preferred embodiments of the present invention an adjustment range for a 
partial coefficient is determined responsive to the magnitude of the quantizers used to quantize 
15 transform coefficients. 

In some preferred embodiments of the present invention the value of at least one 
recovered transform coefficient is adjusted using an adjustment algorithm prior to operating on 
the transform coefficients with a partial transform. In some preferred embodiments of the 
present invention the at least one recovered transform coefficient is adjusted according to an 
20 algorithm responsive to the value of the transform coefficient's quantizer. In some preferred 
embodiments of the present invention, the at least one recovered transform coefficient is 
adjusted according to an algorithm responsive to the value of a recovered transform coefficient 
or the value of partial coefficient. 

According to an adjustment algorithm used in some preferred embodiments of the 
25 present invention, the value of the at least one transform coefficient is adjusted to conform to a 
predetermined pattern relating values of transform coefficients. 

For compressed data representing color images, in some preferred embodiments of the 
present invention, the value of at least one transform coefficient for one color component is 
adjusted responsive to at least one value of a transform coefficient for a different color 
30 component. 

Generally, as in the case of partial coefficients, recovered coefficients generated from a 
first set of compressed data may be adjusted, in accordance with a preferred embodiment of the 
present invention, responsive to recovered coefficients of a second set of compressed data. 
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Using an algorithm to adjust the value of at least one transform coefficient does not 
necessarily change the value of the at least one transform coefficient. In some cases the result 
of using the algorithm is to leave the value of the at least one transform coefficient unchanged. 

The inventors have found that for JPEG compressed images, or images compressed 
using derivative and similar compression techniques, adjusting partial coefficients in 
accordance with a preferred embodiment of the present invention, reduces noise in the 
recovered image to a greater extent than it decreases contributions to the recovered image from 
high spatial frequency transform coefficients. Therefore, the quality of the recovered image is 
enhanced as a result of noise reduction but with less loss of sharpness than usually results from 
noise reduction techniques used in image processing. Furthermore, in accordance with a 
preferred embodiment of the present invention, partial coefficients are adjusted as described 
above. An image recovered using the adjusted partial coefficients is then enhanced with a 
sharpness enhancing procedure, in accordance with some embodiments of the invention. The 
resulting recovered image, in accordance with a preferred embodiment of the present invention, 
is enhanced by both noise reduction and edge enhancement. 

There is therefore provided in accordance with a preferred embodiment of the present 
invention a method for recovering an image defined as a function of image coordinates of an 
image space from compressed data that is a function of transform coordinates in a transform 
space comprising: a) transforming the compressed data using at least one partial transform to 
generate a set of intermediate coefficients in a space intermediate between the transform space 
and the image space; b) adjusting the value of at least one intermediate coefficient and using 
the adjusted value to generate a set of adjusted intermediate coefficients; and c) recovering the 
image by transforming the set of adjusted intermediate coefficients to the image space with at 
least one additional partial transform. 

Preferably, adjusting the value of at least one partial coefficient comprises: a) 
determining an adjustment range; b) determining a value in the adjustment range; and c) setting 
the value of the intermediate coefficient equal to the determined value. 

Preferably, the compressed data comprises information for determining a set of 
quantized values and a quantizer for each quantized value as functions of coordinates of the 
transform space, and transforming the data comprises determining a first set of coefficients in 
the transform space using the quantizers and the quantized values and transforming the first set 
of coefficients with the at least one partial transform. 
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Determining the first set of coefficients preferably comprises determining the product 
of each quantized value with its quantizer and determining the first set of coefficients as a set 
comprising all the products. Alternatively, determining the first set of coefficients preferably 
comprises determining the product of each quantized value with its quantizer, adjusting the 
value of at least one of the products and determining the first set of coefficients as a set 
comprising all the products. 

In some preferred embodiments of the present invention, determining an adjustment 
range comprises determining at least one adjustment limit and determining the adjustment 
range responsive to the adjustment limit. 

Preferably, determining at least one adjustment limit comprises transforming the set of 
quantizers with the at least one transform to generate a set of transformed quantizers in the 
intermediate space and determining an adjustment limit for at least one intermediate coefficient 
responsive to at least one transformed quantizer of the set of transformed quantizers. 

Preferably, determining at least one adjustment limit comprises adjusting the value of at 
least one quantizer responsive to values of the first coefficients prior to transforming the set of 
quantizers. Adjusting the value of at least one quantizer preferably comprises determining 
whether all of the first set of coefficients are zero for a value of a single coordinate of the 
transform space greater than a certain value and constant values for all other transform space 
coordinates and if so, setting the values of the quantizers corresponding to the zero coefficients 
equal to a same value. Preferably, the same value is equal to the maximum of the coefficients 
for all values of the single coordinate and the constant values for all other coordinates. 

Additionally or alternatively, the at least one adjustment limit for the at least one 
intermediate coefficient is preferably determined responsive to the transformed quantizer 
having the same coordinates in the intermediate space as the at least one intermediate 
coefficient. Preferably, the adjustment limit is equal to the corresponding transformed quantizer 
multiplied by a fraction less than one. Preferably, the fraction is equal to 0.5. 

In some preferred embodiments of the present invention, determining at least one 
adjustment limit comprises determining an adjustment limit for at least one intermediate 
coefficient responsive to the value of at least one of the intermediate coefficients. 

Preferably, determining the at least one adjustment limit comprises determining the 
difference between a maximum and minimum intermediate coefficient in a plurality of 
intermediate coefficients in the set of intermediate coefficients and determining the adjustment 
limit responsive to the difference. Preferably, the plurality of intermediate coefficients 
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comprises intermediate coefficients in a neighborhood of the at least one intermediate 
coefficient. 

Alternatively and preferably, all but one of the coordinates of any two of the plurality of 
intermediate coefficients are the same and the plurality of intermediate coefficients includes 
the at least one intermediate coefficient. 

Alternatively or additionally the adjustment limit is preferably equal to the difference 
multiplied by a fraction less than one. Preferably, the fraction is equal to 0.5. 

Some preferred embodiments of the present invention comprise testing the set of 
adjusted intermediate coefficients for consistency with the first set of coefficients in the 
transform space. 

Preferably, testing the set of adjusted intermediate coefficients comprises transforming 
the set of adjusted intermediate coefficients into a second set of coefficients in the transform 
space and comparing coefficients of the first set with coefficients of the second set to determine 
if the set of adjusted coefficients is consistent with the first set of coefficients. Preferably, 
comparing coefficients comprises determining how close the value of at least one coefficient of 
the first set is to the value of at least one coefficient of the second set. Determining how close 
the value of at least one coefficient of the first set is to the value of at least one coefficient of 
the second set preferably comprises using a metric. 

In some preferred embodiments of the present invention, if the set of intermediate 
coefficients is not consistent with the first set of coefficients, the value at least one adjusted 
intermediate coefficient of the set of adjusted intermediate coefficients is readjusted to generate 
a readjusted set of intermediate coefficients so that the set of readjusted intermediate 
coefficients is consistent with the first set of coefficients. 

In some preferred embodiments of the present invention adjusting the at least one 
intermediate coefficient comprises adjusting the at least one intermediate coefficient responsive 
to coefficients in a neighborhood of the at least one intermediate coefficient. Preferably, the 
neighborhood comprises a plurality of intermediate coefficients, including the at least one 
intermediate coefficient, for which any two intermediate coefficients have same values for all 
the same coordinates except one. 

In some preferred embodiments of the present invention, adjusting the at least one 
intermediate coefficient comprises adjusting the at least one intermediate coefficient responsive 
to a ratio between a partial coefficient and a function of other partial coefficients. 
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In some preferred embodiments of the present invention, adjusting the at least one 
intermediate coefficient comprises adjusting the at least one partial coefficient responsive to a 
trend in changes of values of partial coefficients as a function of changes in at least one 
coordinate of the intermediate space. 

In some preferred embodiments of the present invention, adjusting the at least one 
intermediate coefficient comprises adjusting the at least one partial coefficient responsive to an 
expected trend in changes of values of partial coefficients as a function of changes in at least 
one coordinate of the intermediate space. 

In some preferred embodiments of the present invention, adjusting the at least one 
intermediate coefficient comprises adjusting the at least one partial coefficient responsive to a 
predetermined template. Preferably, the template identifies an edge in the recovered image and 
comprising smoothing the recovered image on either side of the identified edge. 

In some preferred embodiments of the present invention, adjusting the value of the at 
least one intermediate coefficient comprises adjusting the value of the at least one intermediate 
coefficient responsive to data from different compressed data that is related to the compressed 
data. Preferably, the image in the image space comprises values for a first color component of a 
color image in the image space and the different compressed data comprises data for generating 
a second color component image of the color image. Additionally or alternatively, the different 
compressed data identify an edge in the recovered image. Some preferred embodiments of the 
present invention in which an edge is identified comprise smoothing the image on either side of 
the edge. 

Some preferred embodiments of the present invention comprise testing the set of 
adjusted intermediate coefficients for consistency with a first set of intermediate coefficients 
defined in an intermediate space different from the intermediate space in which the adjusted 
intermediate coefficients are defined. 

Preferably, testing the set of adjusted intermediate coefficients comprises transforming 
the set of adjusted intermediate coefficients into a second set of intermediate coefficients in the 
different intermediate space and comparing intermediate coefficients of the first set with 
intermediate coefficients of the second set to determine if the set of adjusted coefficients is 
consistent with the first set of intermediate coefficients. 

Preferably, comparing coefficients comprises determining how close the value of at 
least one intermediate coefficient of the first set is to the value of at least one intermediate 
coefficient of the second set. Determining how close the value of at least one intermediate 
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coefficient of the first set is to the value of at least one intermediate coefficient of the second 
set preferably comprises using a metric. 

If the set of adjusted intermediate coefficients is not consistent with the first set of 
intermediate coefficients, in some preferred embodiments of the present invention, the value at 
least one adjusted intermediate coefficient of the set of adjusted intermediate coefficients is 
readjusted to generate a readjusted set of intermediate coefficients so that the set of readjusted 
intermediate coefficients is consistent with the first set of intermediate coefficients. 

There is further provided, in accordance with a preferred embodiment of the present 
invention, a method for recovering an image defined as a function of image coordinates in an 
image space from compressed data that is a function of transform coordinates in a transform 
space the method comprising; a) using the data to generate a first set of recovered transform 
coefficients in the transform space; b) determining an adjustment range for at least one 
recovered transform coefficient; c) replacing the at least one recovered transform coefficient by 
a value in the adjustment range responsive to recovered transform coefficients in a 
neighborhood comprising less than all of the recovered transform coefficients of the at least 
one recovered transform coefficient, to generate an adjusted set of recovered transform 
coefficients; and d) recovering the image by transforming the adjusted set of recovered 
transform coefficients to the image space. 

Preferably, the neighborhood comprises a plurality of recovered transform coefficients 
including the at least one recovered coefficient, for which any two recovered coefficients have 
same values for all the same coordinates except one. 

There is further provided, in accordance with a preferred embodiment of the present 
invention, a method for recovering an image defined in an image space from compressed data 
that comprises quantized values and a quantizer for each quantized value that are functions of 
transform coordinates in a transform space the method comprising: a) determining a set of 
recovered transform coefficients in which each recovered transform coefficient is a product of a 
quantized value with its quantizer; b) determining an adjustment range having an upper and 
lower limit for at least one recovered transform coefficient of the first set of recovered 
coefficients, wherein the difference between the upper and lower limit is less than the 
magnitude of the quantizer used to determine the at least one recovered coefficient; c) replacing 
the at least one recovered transform coefficient by a value in the adjustment range to generate 
an adjusted set of recovered transform coefficients; and d) recovering the image by 
transforming the adjusted set of recovered transform coefficients to the image space. 
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There is further provided, in accordance with a preferred embodiment of the present 
invention, a method for recovering an image defined in an image space from a first set of 
compressed data that comprises quantized values and a quantizer for each quantized value that 
are functions of transform coordinates in a transform space the method comprising: a) 
determining a first set of recovered transform coefficients in which each recovered transform 
coefficient is a product of a quantized value with its quantizer; b) determining an adjustment 
range for at least one recovered coefficient; c) replacing the at least one transform coefficient 
with a value in the adjustment range responsive to recovered transform coefficients generated 
using a second set of related- compressed data to generate an adjusted set of recovered 
transform coefficients; and d) recovering the image by transforming the adjusted set of 
recovered transform coefficients to the image space. Preferably, the first and second sets of 
compressed data are generated from different color component images of a color image. 

There is further provided, in accordance with a preferred embodiment of the present 
invention, a method for adjusting an image having a plurality of image values defined in an 
image space, the image values being generated from compressed data comprising quantized 
values and a quantizer for each quantized value that are functions of transform coordinates in a 
transform space the method comprising: determining an adjustment range for at least one 
image value of the plurality of image values responsive to the quantizers; determining a value 
in the adjustment range; adjusting the image by replacing the at least one image value with the 
determined value. 

Preferably, determining an adjustment range comprises determining an average of a 
plurality of the quantizers. Alternatively, determining an adjustment range preferably 
comprises determining a maximum quantizer in a plurality of the quantizers. Alternatively, 
determining an adjustment range preferably comprises determining a minimum quantizer in a 
plurality of the quantizers. 

In some preferred embodiments of the present invention, the plurality of the quantizers 
comprises all quantizers in the compressed data. 

In some preferred embodiments of the present invention, determining a value comprises 
determining a value responsive to image values of a different related image. Preferably, the 
image and the different related image are color component images of a multicolor color image. 

Some preferred embodiments of the present invention, comprise smoothing the 
recovered image. 
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In some preferred embodiments of the present invention, the image is an image defined 
in a three dimensional image space. 

In some preferred embodiments of the present invention, the image is an image defined 
in a two dimensional space. 
5 The compressed data in some preferred embodiments of the present invention is data 

generated using a unitary separable transform to transform an image in the image space into a 
set of values in the transform space. 

In some preferred embodiments of the present 
generated using a JPEG compression method. 
10 In some preferred embodiments of the present 

generated using an MPEG compression method. 

In some preferred embodiments of the present 
generated using a PX64 compression method. 

In some preferred embodiments of the present 
15 generated using an H261 compression method. 

In some preferred embodiments of the present 
generated using an H263 compression method. 

In some preferred embodiments of the present 
generated using an H323 compression method. 
20 In some preferred embodiments of the present 

generated using an HDTV compression method. 

The invention will be more clearly understood by reference to the following description 
of preferred embodiments thereof read in conjunction with the figures attached hereto. 
BRIEF DESCRIPTION OF FIGURES 
25 Figs. 1 A - IB show a flow diagram schematically illustrating a procedure for recovering 

an image from two dimensional data compressed data, such as JPEG and MPEG compressed 
data, compressed using a two dimensional discrete cosine transform, in accordance with a 
preferred embodiment of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
30 Figs. 1 A and IB show a flow diagram of a procedure 20 for recovering an image from 

two dimensional compressed data 22 (shown in Fig. 1A). Data 22 represents compressed data 
that is compressed from a two dimensional image defined in an image space using any of many 
different suitable transforms and quantization procedures. However, for clarity and efficacy of 



invention, the compressed data is 
invention, the compressed data is 
invention, the compressed data is 
invention, the compressed data is 
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invention, the compressed data is 
invention, the compressed data is 
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exposition it is assumed that data 22 represents data for a color image compressed using a 
JPEG compression procedure. 

Assume, for example, that compressed data 22 represents luminance values of pixels in 
an 8 pixel by 8 pixel tile, of a plurality of pixel tiles into which a color image is partitioned. 
Compressed data 22 therefore comprises information sufficient to determine values for sixty 
four quantized transform coefficients QC(u,v) and a quantizer Q(u,v) for each QC(u,v). It 
should be noted that Q(u,v) is not generally a constant and is often a function of u and v. 

Each quantized transform QC(u,v) is generated from a corresponding transform 
coefficient C(u,v) by quantizing transform coefficient C(u,v) using quantizer Q(u,v). The 
C(u,v) are generated in JPEG by transforming the luminance values of the pixels in the tile 
with the discrete cosine transform. If I(x,y) represents the luminance of the pixel in the x-th 
row and 

7 7 

y-th column of the tile then C(u,v) = £ Z DCT(u,v,x,y)I(x,y), where DCT(u,v,x,y) 

x=0y=0 

represent the two dimensional discrete cosine transform. However since DCT(u,v,x,y) is 

7 7 

separable, the equation for C(u,v) can be written C(u,v) = 2 Z DCX (u,x)DCT_ (v,y)I(x,y) , 

x=0y=0 1 2 

where DCTj(u,x) and DCT2(v,y) are partial transforms that convert image space coordinates, x 

and y, hereinafter referred to as spatial coordinates, into transform space coordinates u and v 

respectively. The inverse discrete cosine transform IDCT(x,y,u,v), can similarly be written as 

IDCT2(y,v)IDCT! (x,u). 

Procedure 20 begins processing compressed data 22 in a step 24 in which compressed 

data 22 is used to generate a data set 26 of recovered transform coefficients RC(u,v). It is often 

convenient to visualize data set 26 as a matrix of elements whose values are the magnitudes of 

the RC(u,v), where u is the row and v is the column of the matrix in which the value of RC(u,v) 

is located. Following common convention the element in the upper left corner of the matrix is 

the magnitude of RC(0,0). Data sets comprising the QC(u,v), Q(u,v) and other related data sets 

are similarly conveniently visualized. An RC(u,v) matrix 28 is schematically shown in inset 30 

of Fig. 2 

In step 32 a decision is made whether or not to adjust the recovered transform 
coefficients RC(u,v). In some preferred embodiments of the present invention the decision to 
adjust or not to adjust is made responsive to values of RC(u,v) and/or values of Q(u,v). For 
example in accordance with a preferred embodiment of the present invention only values of 
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RC(u,v) that are equal to ± 1 are adjusted, since relative errors due to quantization in these 
recovered coefficients are generally greater than relative errors in higher valued recovered 
coefficients. 

If procedure 20 does adjust recovered transform coefficients RC(u,v), procedure 20 
advances to step 34 in which the value of at least one recovered transform coefficient RC(u,v) 
is adjusted using at least one adjustment algorithm. 

According to one adjustment algorithm, in accordance with a preferred embodiment of 
the present invention, the value of RC(u,v) is set equal to [QC(u,v)Q(u,v) - Q(u,v)(u+v)/32] 
instead of QC(u,v)Q(u,v)._ A recovered transform coefficient RC(u,v) can differ from the 
transform coefficient C(u,v) from which it was quantized by an amount that can vary up to a 
maximum amount, hereinafter referred to as a "quantization error", that is one half the 
magnitude of the quantizer used to quantize C(u,v). Therefore, the effect of this algorithm is to 
bias the value of a recovered transform coefficient RC(u,v) to lower values of a range of 
possible values for the transform coefficient C(u,v) from which QC(u,v) was generated by an 
amount equal to a fraction of its quantization error. Furthermore the bias is larger for higher 
spatial frequency recovered transform coefficients than for lower spatial frequency recovered 
transform coefficients {i.e. for C(u,v) with larger values for u+v). In a variation of this 
algorithm, RC(u,v) is set equal to [QC(u,v)Q(u,v) - ocQ(u,v)], where a is a fraction less than 
one. According to this algorithm for adjusting recovered transform coefficients, each 
recovered transform coefficient is reduced by the same fraction of its quantization error. 
Preferably a is greater than 0.5. More preferably, a is equal to 0.75. 

Generally, the magnitudes of cosine transform coefficients of an image decrease 
substantially monotonically with increasing spatial frequency, especially for relatively small 
homogeneous images. If recovered transform coefficients RC(u,v) do not exhibit this expected 
behavior they are adjusted to reduce deviation from the expected behavior using another 
adjustment algorithm in accordance with a preferred embodiment of the present invention. In 
accordance with a preferred embodiment of the present invention, RC(u,v) are increased or 
decreased by amounts up to their quantization errors, so that to an extent possible, for a 
constant value for one of the arguments u or v, the absolute values of RC(u,v), or alternatively 
the values of RC(u,v), monotonically decrease with increase in the other argument. In some 
preferred embodiments of the present invention RC(u,v) that are equal to zero are not included 
in the adjustment procedure. 
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For example, a straight line can be fitted by regression to the RC(u,v) in a row. If the 
regression line has a negative slope, individual RC(x,v) are adjusted within their quantization 
errors so that they are as close as possible to the regression line. If the regression line does not 
have a negative slope, values of RC(x,v) are not adjusted. Alternatively, for example, if the 
5 regression line does not have a negative slope, a regression line with a negative slope can be 
searched for using a subset of the RC(x,v). If a suitable regression line is found then values of 
the RC(x,v) in the subset are adjusted to exhibit monotonically decreasing values. For example, 
a regression curve can be fitted only for those RC(x,v) having spatial frequencies greater than 
the spatial frequency of the maximum RC(x,v) in the row. Other ways of defining a negative 

1 0 slope for adjusting the R(x,v) will occur to persons of the art. 

The decrease in magnitude of transform coefficients with increasing spatial frequency is 
a characteristic common to many transforms. The transform coefficients can be considered 
projections of an image on basis functions of the transform. The basis functions exhibit 
different rates of spatial variation and can be arranged in an ordered set in which each basis 

15 function in the set exhibits a higher rate, i.e. frequency, of spatial variation than the basis 
function that precedes it in the ordered set. The first basis function, often called the DC 
function, is a constant and its corresponding transform coefficient is an average of the image 
over a spatial range in which the image is defined. At some basis function spatial frequency, 
the frequency of variation of a basis function is generally larger than a dominant rate of change 

20 in the image. From that spatial frequency on, the magnitudes of the transform coefficients are 
generally monotonically decreasing with increasing spatial frequency. For relatively 
homogeneous images (the tiles in a JPEG image are generally small enough so that they are 
substantially homogeneous) the transform coefficients are substantially monotonically 
decreasing from the DC transform coefficient on. 

25 The application of an adjustment algorithm to data set 26 of recovered transform 

coefficients RC(u,v) generates a data set 36 of adjusted recovered transform coefficients 
RC*(u,v). Procedure 20 then advances to a step 38 in which data set 36 is transformed by one 
of the partial inverse discrete cosine transforms, IDCTi (x,u) or IDCT2(y,v), into a data set 40 
of partial coefficients PRC(x,v) or PRC(u,y). For clarity and economy of presentation it is 

30 assumed that in step 38, data set 36 comprising RC*(u,v) is transformed using IDCT](x,u) so 
that data set 40 comprises partial coefficients PRC(x,v). 

If the decision in decision step 32 is not to adjust at least one recovered transform 
coefficient RC(u,v) then procedure 20 advances directly from decision step 32 to step 38 and 
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partial transform EDCTi(x,u) transforms data set 26 into data set 40 instead of transforming 
data set 36 into data set 40. 

Transforming data set 40 of partial coefficients PRC(x,v) with partial transform 
DCTi(u,x) regenerates the RC(u,v), or the RC*(u,v), from which the PRC(x,v) are generated. 

A visualization of PRC(x,v) data set 40 as a matrix 42 is schematically shown in inset 
44 of Fig. 1 . The spatial coordinate x indicates the row and the transform coordinate v indicates 
the column of matrix 42 in which an element PRC(x,v) is located. PRC(0,0) is in the upper left 
corner of matrix 42. If recovered transform coefficients RC(u,v) were equal to transform 
coefficients C(u,v), i.e. if there is no error or information loss in compressed data 26, and no 
RC(u,v) is adjusted, then the PRC(x,v) in a row "X" would be the cosine transform of I(x,y) for 
x = X along the y-axis. Increasing column number v corresponds to increasing spatial 
frequency along the y-axis of I(x,y). 

Following generation of data set 40 of partial coefficients PRC(x,v), in step 46, an 
adjustment range is preferably determined for at least one PRC(x,v). An adjustment range for a 
PRC(x,v) can be determined in different ways, in accordance with various preferred 
embodiments of the present invention. 

Assume PRC(x,v) has an adjustment limit, ALIM(x,v). An adjustment algorithm, in 
accordance with a preferred embodiment of the present invention, adjusts PRC(x,v) by 
assigning PRC(x,v) a value in an adjustment range of values defined by PRC(x,v) ± 
ALIM(x,v). Whereas the noted defined range of values is symmetric, it is possible, and may be 
advantageous, in accordance with a preferred embodiment of the present invention, to define a 
range that is asymmetric about PRC(x,v). Furthermore, it should be noted that adjustment 
limits and adjustment ranges can be computed in different ways and have different values for 
different adjustment algorithms. 

In accordance with a preferred embodiment of the present invention adjustment limits 
for PRC(x,v) are determined by transforming the quantizers Q(u,v) of compressed data 22 with 
IDCTj(x,u) to generate a set of partial quantizers PQ(x,v). ALIM(x,v) is then preferably set 
equal to a(x,v)PQ(x,v), where a(x,v) is preferably a fraction less than or equal to one half. 
More preferably a is set equal to one half. 

In some preferred embodiments of the present invention the Q(u,v) are adjusted to 
provide a set of adjusted quantizers Q*(u,v) that are used instead of Q(u,v) to calculate the 
ALIM(x,v). The Q*(u,v) are not used to calculate recovered coefficients RC(u,v) only the 
values for adjustment ranges. 
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In some preferred embodiments of the present invention, adjustments to Q(u,v) are 
made responsive to values of quantized transform coefficients QC(u,v) in compressed data 22. 
For example, assume that quantized transform coefficients QC(u,v) are equal to zero for v > V 
in a row u = U. Assume that the maximum value for the QC(u,v) in row U occurs in column W, 
so that the maximum quantized transform coefficient in row U is QC(U,W). Then, in 
accordance with a preferred embodiment of the present invention, the quantizers Q(u,v) for u = 
U and v > V are set equal to QC(U,W) to generate an adjusted set of quantizers Q*(u,v) with 
which to determine adjustment limits ALIM(x,v). Since it is expected that QC(u,v) decrease 
monotonicaliy with v it is reasonable to limit an adjustment limit for the QC(U,v) that are zero 
to the value of the largest non-zero QC(U,v). 

Adjustment limits ALIM(x,v) can also be determined responsive to values of partial 
coefficients PRC(x,v), in accordance with a preferred embodiment of the present invention. For 
example, an adjustment limit ALIM(x,v) for x = X and v = V can be determined to be equal to 
a fraction less than one of the difference between a maximum and a minimum partial 
coefficient PRC(x,v) in row X or column V or a surround of "location" X,V. Preferably the 
fraction is greater than or equal to 0.5. More preferably the fraction is equal to 0.5. Other 
methods for determining adjustment limits ALIM(x,v) will occur to persons of the art. 

In yet other preferred embodiments of the present invention ALIM(x,v) is determined to 
be equal to a product of a fraction less than one times the minimum PRC(x,v) in either row x or 
column v. 

Following determination of an adjustment limit ALIM(x,v) and an adjustment range in 
step 46 for at least one PRC(x,v), the at least one PRC(x,v) is then adjusted, in a step 48, using 
at least one appropriate adjustment algorithm. The adjustment algorithm is used to determine a 
new value for each of the at least one PRC(x,v), which new value lies within the adjustment 
range of the PRC(x,v) determined from the adjustment limit ALIM(x,v). The value of the 
PRC(x,v) is then set equal to the new value. The result of adjusting the at least one PRC(x,v) is 
generation of an adjusted data set 50 of partial transform coefficients PRC*(x,v). 

Whereas the at least one adjustment algorithm is described as used subject to the 
constraint of an adjustment limit, it is possible, in accordance with a preferred embodiment of 
the present invention, to use algorithms that are not constrained by an adjustment limit. 

It should be noted that a new value for a PRC(x,v) determined by an algorithm can be 
equal to an "old" value of the PRC(x,v) and that therefore in some circumstances, in 
accordance with a preferred embodiment of the present invention, some or all values in data set 
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50 of adjusted transform coefficients PRC*(x,v) are identical to corresponding values of data 
set 40 of transform coefficients PRC(x,v). 

According to one adjustment algorithm, in accordance with a preferred embodiment of 
the present invention, PRC(x,v) in at least one row (i.e. constant x) are increased or decreased 
by amounts limited by at least one adjustment limit so that to an extent possible the values 
PRC(x,v) decrease monotomcally with increasing column number (i.e. increasing v). This can 
be achieved, for example, using a regression line determination and fitting technique similar to 
the one described above for adjusting recovered transform coefficients RC(u,v) so that the 
values of the recovered transform coefficients decrease substantially monotonically with 
increasing column or row number. 

According to another adjustment algorithm, in accordance with a preferred embodiment 
of the present invention, PRC(x,v) in at least one row are adjusted to moderate an anomalous 
value of a PRC(x,v) in the row. In some preferred embodiments of the present invention the 
value of the PRC(x,v) is tested for anomaly by determining an average of values of PRC(x,v) 
in the row and comparing the value of the PRC(x,v) with the average. For example, if AV 
represents the determined average and ALIM(x,v) represents a symmetric adjustment limit for 
the PRC(x,v) (i.e. PRC(x,v) is adjustable in a range PRC(x,v)±ALIM(x,v)), then if j(PRC(x,v) 
- AV)| > ALIM(x,v) the value of the PRC(x,v) is considered anomalous. In some preferred 
embodiments of the present invention when a PRC(x,v) is anomalous, a low pass filter is 
applied to the PRC(x,v) and adjacent PRC(x,v) in the row in order to moderate the anomaly. In 
some preferred embodiments of the present invention, the value of an anomalous PRC(x,v) is 
multiplied by an appropriate factor less than one to reduce the anomaly. In some preferred 
embodiments of the present invention an average AV is calculated according to 
M 

AV = (1/2M)* Z[PRC(x,v-m) + PRC(x, v + m)] . 
m = l 

In some preferred embodiments of the present invention anomalies that are moderated 
are anomalies that are identified in a value of a first PRC(x,v) by comparing the value of the 
first PRC(x,v) with values of other PRC(x,v) that are in the same column as the first PRC(x,v). 
In some preferred embodiments of the present invention the value of a first PRC(x,v) is 
compared to values of other PRC(x,v) in an area of the matrix surrounding the first PRC(x,v) to 
determine if the value of the first PRC(x,v) is anomalous. 

According to another adjustment algorithm, in accordance with a preferred embodiment 
of the present invention, the value of a particular PRC(x,v) is adjusted responsive to a ratio 
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between other PRC(x,v). For example, generally values of PRC(x,v) for column numbers 
greater than 2 are less reliable than values of PRC(x,v) for column numbers less than or equal 
to two. This occurs, generally, because higher spatial frequency transform coefficients C(u,v) 
are generally smaller than lower frequency transform coefficients C(u,v) and because the 
5 higher frequency transform coefficients are usually quantized with larger quantizers than lower 
spatial frequency transform coefficients C(u,v). Accordingly, in some preferred embodiments 
of the present invention, PRC(x,v) in higher number columns are adjusted in response to a ratio 
between a higher column number PRC(x,v) and a value that is determined form lower column 
number PRC(x,v), 

10 For example, assume that PRC(x,v) for a column v = V are to be adjusted and that V > 

3. According to a preferred embodiment of the present invention a row X is determined for 
which the value PRC(x,V) is a maximum. A value "VAL(X)" is set equal to the average of, 
PRC(X,0) and PRC(X,1) in row X, and an adjustment ratio AR = PRC(X,V)/VAL(X) is 
calculated. The value of at least one other PRC(x,V) in column V is then adjusted by replacing 

15 it with a new value "NPRC(x,V)" that satisfies the relation[PRC(x,V) - ALIM(x,V)] < 
NPRC(x,V)< [PRC(x,V) + ALIM(x,V)] and minimizes [NPRC(x,V) - VAL(x)AR]. In these 
relations VAL(x) is the average of PRC(x,0) and PRC(x,l) and ALIM(x,V) is the adjustment 
limit of PRC(x,V). 

According to another adjustment algorithm, in accordance with a preferred embodiment 
20 of the present invention, PRC(x,v) in a same row X are set to zero if the value PRC(X,v) 
satisfies the relation: ALIM(X,v) < PRC(X,v) < +ALIM(X,v) for all v. In some preferred 
embodiments of the present invention, if the above relation is satisfied by all but one or two of 
the PRC(X,v) then a low pass filter is applied to PRC(X,v) to moderate variations in PRC(X,v). 
According to other adjustment algorithms, in accordance with preferred embodiments 
25 of the present invention, a partial coefficient or a transform coefficient for one color component 
image of a color image is adjusted in response to at least one transform coefficient or partial 
coefficient of a different color component image of the color image. 

For example in JPEG, chrominance components U and V are often coded for every 
other pixel of a color image while luminance Y is coded for every pixel, i.e. Y is sampled at 
30 twice the rate as U and V in the color image. Typically, for an 8 by 8 tile of pixels in the color 
image, Y will have 64 values while U and V will each have only sixteen values. As a result, 
luminance Y will be specified by sixty-four recovered transform coefficients for the tile while 
U and V will each be specified by sixteen recovered transform coefficients for the same tile. 
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Each row or column of recovered luminance transform coefficients Y comprises eight 
coefficients while each row or column of recovered chrominance transform coefficients, U or 
V, comprises only four coefficients. Recovered Y transform coefficients for the tile will 
therefore comprise sixty four recovered coefficients and recovered U and V transform 
coefficients will comprise 16 recovered coefficients. Using an adjustment algorithm, in 
accordance with a preferred embodiment of the present invention, the four recovered transform 
coefficients for a chrominance component are extrapolated to eight transform coefficients, 
responsive to values of the chrominance coefficients and values of the luminance coefficients. 
Extrapolating the. four chrominance coefficients to eight chrominance coefficients provides an 
estimate of chrominance coefficients that might result from a finer sampling of the 
chrominance component of the image. 

In accordance with some preferred embodiments of the present invention, recovered U 
coefficients (or, of course, recovered V coefficients or both recovered U and V coefficients) are 
extrapolated to eight recovered U coefficients that have values that reflect a trend in the values 
of the Y coefficients. For example, assume that a row of Y recovered coefficients for a tile of a 
color image have values 100, 60, 30, 30, 20, 10, 0, 0 and that corresponding U recovered 
coefficients for the tile have values 60, 40, 10, 0. Then an extrapolated set of eight recovered U 
coefficients, in accordance with a preferred embodiment of the present invention might have 
values 70, 50, 40, 40, 15, 5, 0, 0. 

In accordance with some preferred embodiments of the present invention an edge that is 
well defined by data for one color component image of a color image is used to locate an edge 
in recovered coefficients and/or partial coefficients or in a recovered image of another color 
component image of the color image. Once the edged is located, in some preferred 
embodiments of the present invention the recovered coefficients and/or the partial coefficients 
and/or the recovered image are smoothed on either side of the located edge. This preserves the 
edge and thereby sharpness of the recovered image while reducing noise in the recovered 
image. 

Whereas the above example is given for recovered transform coefficients, the example 
applies equally well to adjusting a partial transform coefficient for one color component using 
recovered partial coefficients or transform coefficients for a different color component. 

Furthermore, adjusting recovered coefficients and/or partial coefficients for one set of 
compressed data responsive to recovered coefficients and/or partial coefficients in a related set 
of compressed data is not limited to color component images of color images, in accordance 
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with a preferred embodiment of the present invention. Quite generally, recovered coefficients 
and or partial coefficients generated from a first set of compressed data may be adjusted, in 
accordance with, a preferred embodiment of the present invention, responsive to recovered 
coefficients and/or partial coefficients of a related second set of compressed data. 

According to other adjustment algorithms, in accordance with preferred embodiments 
of the present invention, the value of at least one partial coefficient is adjusted to conform to a 
predetermined pattern, hereinafter referred to as a "template", that defines relationships 
between a set of partial coefficients. 

A template comprises a set of values and a variance for each value, hereinafter referred 
to respectively as a "template" value and a "template" variance. Each template value and its 
template variance defines a range of values for a different partial coefficient of a set of partial 
coefficients. The template values are compared, in accordance with a preferred embodiment of 
the present invention, to the set of partial coefficients using a metric responsive to template 
variance and adjustment limit of the partial coefficients. The comparison is used to determine 
to what degree relationships between the template values and relationships between the partial 
coefficients are similar. If, according to the metric, the template and the set of partial 
coefficients are considered to match, i.e. relationships between the template values and 
relationships between the partial coefficients are sufficiently similar, then the partial 
coefficients are adjusted to improve the match. 

For example, a template for row x = X of JPEG partial coefficients i.e. PRC(X,v), 
might comprise eight template values and variances that are functions of column number v. For 
the row, let the template values be represented by T(X,v) and the template variances be 
represented by TALIM(X,v). Then, in accordance with a preferred embodiment of the present 
invention, a suitable "matching" metric to determine the degree to which the row of partial 
coefficients RC(X,v) and the template values T(X,v) match, is 

"MATCH" =£[PRC(X,v)-<2rT(X, v )] 2 /[TALIM(X,v) 2 +ALIM(X, v) 2 ]. In the equation, a 
v 

is a suitable scaling constant that best removes any constant factor between values of PRC(X,v) 
and T(X,v). If MATCH is less than an appropriate threshold, then the PRC(X,v) and the T(X,v) 
are considered a match. If there is a match then, in accordance with a preferred embodiment of 
the present invention, at least one value of the PRC(X,v) is adjusted within limits determined 
by its adjustment limit R(X,v) to minimize MATCH. 
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In accordance with some preferred embodiments of the present invention, a template 
matching algorithm is used to test a set of partial coefficients for a match with each of a 
plurality of different templates to determine which, if any, of the templates best matches the set 
of coefficients. In some preferred embodiments of the present invention the best matching 
template is chosen for use in adjusting the partial coefficients. For example, in the JPEG 
example, MATCH might be calculated for each of a plurality of different templates and the 
template that provides the smallest value for MATCH will be the template used to adjust at 
least one PRC(X,v). 

In other preferred embodiments of the present invention, if more than one template is 
found to match a set of partial coefficients, as determined by a suitable matching metric, all of 
the matching templates are used to adjust at least one of the partial coefficients. Preferably, this 
is performed by forming a weighted average of the matching templates and using values of the 
weighted average to adjust at least one of the partial coefficients. For example, assume that 
"m" templates Tj(X,v) are found to match the JPEG partial coefficients PRC(X,v). If a 
weighted average of the matching templates is represented by WT(X,v) then WT(X,v) 
m 

= £ /?.T.(X,v) where are appropriate weighting factors and each PRC(X,v) is preferably 
i = 0 1 1 

adjusted responsive to WT(X,v). 

Preferably, in accordance with a preferred embodiment of the present invention, some 

of the templates that are tested for a match with a set of recovered transform coefficients are 

templates that correspond to possible edges in the image from which the compressed data is 

generated. When a match is found between a set of partial coefficients and a template that 

represents an edge, the match results in identifying the position of an edge in an image 

recovered using the partial coefficients. In accordance with a preferred embodiment of the 

present invention, pixels in the recovered image on either side of the identified edge are then 

adjusted using a low pass filter that does not operate simultaneously on pixels on both sides of 

the edge. This procedure reduces noise in the recovered image without reducing sharpness in 

the image. 

Whereas the above example is given for partial coefficients, the example applies 
equally well to adjusting recovered transform coefficients and adjusting recovered pixel values. 

Variations of the adjustment algorithms discussed above and other algorithms for 
adjusting partial transform coefficients will occur to persons of the art. 
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In a preferred embodiment of the present invention, following generation of adjusted 
data set 50, data set 50 is preferably transformed "back", in a step 52 with partial transform 
- DCTi (u,x) to generate a data set 54 of regenerated transform coefficients RC**(u,v). Values of 
RC**(u,v) are checked for consistency with values of RC*(u,v) or RC (u,v) in a step 56 by 
5 comparing values of RC**(u,v) with values of RC*(u,v) or RC (u,v) using an appropriate 
metric. Preferably, the metric is responsive to differences between values of RC**(u,v) and 
values of RC(u,v) or RC(u,v) and relevant quantization errors and/or adjustment ranges. For 
example, in some preferred embodiments of the present invention, consistency requires that 
2|RC**(u,v) - RC(u,v)|/Q(u,v) < 1 (Q(u,v) is the quantizer for RC(u,v)). 

10 In decision step 58, if the check for consistency is negative, then procedure 20 returns 

to step 48 and a new value for at least one PRC*(x,v) of data set 50 is determined using 
appropriate adjustment algorithms. For example, in accordance with a preferred embodiment of 
the present invention a new value for a PRC*(x,v) is determined by subtracting from 
PRC*(x,v) an amount equal to a[PRC*(x,v) - PRC(x,v)] where a is a fraction less than one. 

15 Data set 50 is then again preferably checked for consistency with RC*(u,v) or RC (u,v). 
Preferably, new values for at least one PRC*(x,v) of data set 50 are determined until data set 
50 satisfies the consistency check performed in step 56. 

If the check for consistency is positive, decision step 58 preferably advances procedure 
20 to a step 60 (shown in Fig. 2B). In step 60 RC**(u,v) data set 54 is transformed to a data set 

20 62 of partial coefficients PRC(u,y) using IDCT2(y,v). Data set 62 is then preferably adjusted 
and checked in a sequence of steps represented by a process step 64 to produce a data set 66 of 
adjusted partial coefficients PRC*(u,y). Process step 64 represents a series of step similar to the 
series of steps used to adjust and check data set 40 comprising PRC(x,v) to produce data set 50 
comprising PRC*(x,v). However data set PRC(u,y) can be checked for consistency, in 

25 accordance with a preferred embodiment of the present invention, not only with RC(u,v) and/or 
RC*(u,v) but also with data set 50 of adjusted partial coefficients PRC*(x,v). 

In step 68 data set 66 is transformed with partial inverse transform IDCTi(x,u) to 
generate recovered image 70 comprising pixel values RI(x,y). 

The inventors have found that processing compressed data 22 in accordance with a 

30 preferred embodiment of the present invention selectively suppresses noise in recovered image 
70 while affecting sharpness of recovered image 70 only moderately. As a result recovered 
image 70 can be edge enhanced by adjusting pixel values RI(x,y) so as to provide an image that 
has improved sharpness and reduced noise. Therefore, preferably, procedure 20 comprises a 
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step 72 in which recovered image 70 is edge enhanced to provide a sharpened recovered image 
74 comprising pixel values RI*(x,y). 

In adjusting pixel values RI(x,y) of image 70 to generate enhanced image 74, preferably 
an adjustment limit is defined for each pixel value RI(x,y) that is adjusted, which adjustment 
limit is used to define an adjustment range for the pixel. Each pixel values RI*(x,y) of 
enhanced image 74 is preferably constrained to have a value that lies within the adjustment 
range determined for its corresponding pixel value RI(x,y) in recovered image 70. Preferably 
adjustment limits for pixel values RI*(x,y) are defined responsive to Q(u,v). In some preferred 
embodiments of the present invention an adjustment limit for RI(x,y) is determined to be equal 
to an average of values of all or a subset of Q(u,v) multiplied by a fraction less than one. In 
some preferred embodiments of the present invention an adjustment limit is set equal to a 
maximum or a minimum value of all or a subset of Q(u,v) multiplied by a fraction less than 
one. Preferably the fraction in either case is less than one half. 

It should be noted that, in accordance with a preferred embodiment of the present 
invention, it is not required to generate any particular combination of data sets to generate a 
recovered image from compressed data. For example, in generating recovered image 70 from 
compressed data 22 it is possible, in accordance with a preferred embodiment of the present 
invention, to generate and adjust only data sets 26 and 36. Recovered image 70 is then 
generated directly from data set 36 without generating and adjusting partial coefficients. It is 
also possible, in accordance with a preferred embodiment of the present invention, to generate 
recovered image 70 directly from data set 50 without generating data sets 62 or 66. 

Quite generally, in accordance with preferred embodiments of the present invention, 
different combinations of transforms and inverse transforms and permutations of a combination 
of transforms and inverse transforms can be used to process compressed data. Furthermore, in 
processing compressed data in accordance with a preferred embodiment of the present 
invention, any data set generated from the compressed data can be adjusted responsive to a 
consistency check with any other data set generated from the compressed data (including a set 
of recovered transform coefficients generated from the data set). 

For example, in accordance with preferred embodiments of the present invention 
different combinations of inverse transforms IDCT] and IDCT2 and transforms DCTj and 
DCT2can be used in processing compressed data 22 It is possible to apply the transform 
IDCTi(x,u>DCT2(v,y) to data set 66 comprising PRC*(u,y) to check the consistency of 
PRC*(u,y) with adjusted partial coefficients PRC*(x,v) of data set 50. It is also possible to 
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transform recovered image 70 of pixel values RI(x,y), or image 74 comprising pixel values 
RI*(x,y) with DCTi(u,x)-DCT2(v,y) and check the resulting data set for consistency with data 
set 26 comprising the "original" recovered transform coefficients RC(u,v). 

It should also be noted that different adjustment algorithms can be applied, in 
accordance with a preferred embodiment of the present, to different data sets generated from 
the compressed data and that more than one adjustment algorithm can be applied to the same 
data set. 

Whereas the invention has been described for a two dimensional image compressed 
using JPEG, the invention applies similarly to n-dimensional images in general and to various 
different types of n-dimensional images including non-video images such as sound files. The 
invention also applies to n-dimensional images that are compressed using a separable unitary 
transform other than a cosine transform. 

A preferred embodiment of the present invention is described in a provisional 
application entitled "Enhancement of Compressed Images" and appendix thereof submitted to 
the US Patent Office on Jan 13,1999, in the name of the same inventors of the present 
invention, the disclosure of which is incorporated herein by reference. 

In the claims and description of the present application, each of the verbs, "comprise", 
"include" and has, and conjugates thereof, are used to indicate that the object or objects of the 
verb are not necessarily a complete listing of all the components, elements or parts of the 
subject or subjects of the verb. 

The present invention has been described using non-limiting detailed descriptions of 
preferred embodiments thereof that are provided by way of example and are not intended to 
limit the scope of the invention. Variations of embodiments described will occur to persons of 
the art. The scope of the invention is limited only by the following claims: 
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1. A method for recovering an image defined as a function of image coordinates of an 
image space from compressed data that is a function of transform coordinates in a transform 
space comprising: 

a) transforming said compressed data using at least one partial transform that transforms 
at least one bnt not all of the transform coordinates into a corresponding image coordinate to 
generate a set of intermediate coefficients which are functions of at least one coordinate from 
the image space and at least one coordinate from the transform space in a space intermediate 
between said transform space and said image space said intermediate space being defined by at 
least one coordinate from the image space and at least one coordinate from the transform space; 

b) adjusting the value of at least one intermediate coefficient and using said adjusted 
value to generate a set of adjusted intermediate coefficients; and 

c) recovering said image by transforming said set of adjusted intermediate coefficients 
to said image space with at least one additional partial transform. 

2. A method according to claim 1 wherein adjusting the value of at least one partial 
coefficient comprises: 

a) determining an adjustment range; 

b) deteraiining a value in said adjustment range; and 

c) setting the value of said intermediate coefficient equal to said determined value. 

3. A method according to claim 2 wherein said compressed data comprises information for 
determining a set of quantized values and a quantizer for each quantized value as functions of 
coordinates of said transform space, and wherein Irarisforming said data comprises deteirnining 
a first set of coefficients in said transform space using said quantizers and said quantized values 
and transforming said first set of coefficients with said at least one partial transform. 

4. A method according to claim 3 wherein detennming said first set of coefficients 
comprises determining the product of each quantized value with its quantizer and deterrnining 
said first set of coefficients as a set comprising all said products. 
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5. A method according to claim 3 wherein detennining said first set of coefficients 
comprises dsterxaxntig the product of each quantized value with its quantizer, adjusting the 
value of at least one of said products and detenninhig said first set of coefficients as a set 
comprising all said products. 

5 

6. A method according to any of claims 3-5 wherein determining an adjustment range 
comprises deterrnining at least one adjustment limit and determining said adjustment range 
responsive to said adjustment limit. 

10 7. A method according to claim 6 wherein determining at least one adjustment limit 
comprises transforming said set of quantizers with said at least one transform to generate a set 
of transformed quantizers in said intermediate space and deterniining an adjustment limit for at 
~_ least one intermediate coefficient responsive to at least one transformed quantizer of said set of 

transformed quantizers. 

: 15 

8. A method according to claim 7 comprising adjusting the value of at least one quantizer 
responsive to values of said first coefficients prior to transforming said set of quantizers. 

9. A method according to claim 8 wherein adjusting the value of at least one quantizer 
20 comprises deterrnining whether all of said first set of coefficients are zero for a value of a 

single coordinate of said transform space greater than a certain value and constant values for all 
other transform space coordinates and if so, setting the values of the quantizers corresponding 
to said zero coefficients equal to a same value. 

25 10. A method according to claim 9 wherein said same value is equal to the maximum of 
said coefficients for all values of said single coordinate and said constant values for all other 
coordinates. 

11. A method according to any of claims 7-10 wherein said at least one adjustment limit 
30 for said at least one intermediate coefficient is determined responsive to the transformed 
quantizer having the same coordinates in the intermediate space as the at least one intermediate 
coefficient. 



30 



AMENDED SHEET 



24 - 11 - 2OOC W07A02 



IL 0099000 



12. A method according to claim 11 wherein said adjustment limit is equal to said 
corresponding transformed quantizer multiplied by a fraction less than one. 

13. A method according to claim 12 wherein said fraction is equal to 0.5. 

5 

14. A method according to claim 6 wherein detennining at least one adjustment limit 
comprises determining an adjustment limit for at least one intermediate coefficient responsive 
to the value of at least one of said intermediate coefficients. 



15. A method according to claim 14 wherein deterrnining said at least one adjustment limit 
comprises determining the difference between a maximum and minimum intermediate 
coefficient in a plurality of intermediate coefficients in said set of intermediate coefficients and 
detennining said adjustment limit responsive to said difference. 

16. A method according to claim 15 wherein said plurality of intermediate coefficients 
comprises intermediate coefficients in a neighborhood of said at least one intermediate 
coefficient. 



17. A method according to claim 15 wherein all but one of the coordinates of any two of 
20 the plurality of intermediate coefficients are the same and wherein the plurality of intermediate 
coefficients includes the at least one intermediate coefficient 



18. A method according to any of claims 15 - 17 wherein said adjustment limit is equal to 
said difference multiplied by a fraction less than one. 

25 

19. A method according to claim 1 6 wherein said fraction is equal to 0.5. 

20. A method according to any of claims 3-19 comprising testing said set of adjusted 
intermediate coefficients for consistency with said first set of coefficients in said transform 

30 space. 

21. A method according to claim 20 wherein testing said set of adjusted intermediate 
coefficients comprises transforming said set of adjusted intermediate coefficients into a second 
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set of coefficients in said transform space and comparing coefficients of said first set with 
coefficients of said second set to determine if said set of adjusted coefficients is consistent with 
said first set of coefficients. 

5 22. A method according to claim 21 wherein comparing coefficients comprises determining 
how close the value of at least one coefficient of said first set is to the value of at least one 
coefficient of said second set. 

23. A method according to claim 22 wherein determining how close the value of at least 
10 one coefficient of said first set is to the value of at least one coefficient of said second set 

comprises using a metric. 

24. A method according to any of claims 20 - 23 wherein, if said set of intermediate 
coefficients is not consistent with said first set of coefficients, the value at least one adjusted 

15 intermediate coefficient of said set of adjusted intermediate coefficients is readjusted to 
generate a readjusted set of intermediate coefficients so that said set of readjusted intermediate 
coefficients is consistent with said first set of coefficients. 

25. A method according to any of the preceding claims wherein adjusting said at least one 
20 intermediate coefficient comprises adjusting said at least one intermediate coefficient 

responsive to coefficients in a neighborhood of said at least one intermediate coefficient 

26. A method according to claim 25 wherein said neighborhood comprises a plurality of 
intermediate coefficients, including said at least one intermediate coefficient, for which any 

25 two intermediate coefficients have same values for all the same coordinates except one. 

27. A method according to any of the preceding claims wherein adjusting said at least one 
intermediate coefficient comprises adjusting said at least one intermediate coefficient 
responsive to a ratio between a partial coefficient and a function of other partial coefficients. 

30 

28. A method according to any of the preceding claims wherein adjusting said at least one 
intermediate coefficient comprises adjusting said at least one partial coefficient responsive to a 
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tread in changes of values of partial coefficients as a fonction of changes in at least one 
coordinate of said intermediate space. 

29. A method according to any of the preceding claims wherein adjusting said at least one 
5 intermediate coefficient comprises adjusting said at least one partial coefficient responsive to 

an expected trend in changes of values of partial coefficients as a function of changes in at least 
one coordinate of said intermediate space. 

30. A method according to any of the preceding claims wherein adjusting said at least one 
10 intermediate coefficient comprises adjusting said at least one partial coefficient responsive to a 

: predetermined template. 

31. A method according to claim 30 wherein said template identifies an edge in said 
recovered image and comprising smoothing said recovered image on either side of said 

15 identified edge. 

32. A method according to any of the preceding claims wherein adjusting the value of said 
at least one intermediate coefficient comprises adjusting the value of said at least one 
intermediate coefficient responsive to data from different compressed data that is related to said 

20 compressed data. 

33. A method according to claim 32 wherein said image in said image space comprises 
values for a first color component of a color image in said image space and said different 
compressed data comprises data for generating a second color component image of said color 

25 image. 

34. A method according to claim 32 or claim 33 wherein said different compressed data 
identify an edge in said recovered image. 

30 35. A method according to claim 3 1 or claim 34 comprising smoothing said image on either 
side of said edge. 
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36. A method according to any of the previous claims comprising testing said set of 
adjusted intermediate coefficients for consistency with a first set of intermediate coefficients 
defined in an intermediate space different from said intermediate space in which said adjusted 
intermediate coefficients axe defined. 

5 

37. A method according to claim 36 wherein testing said set of adjusted intermediate 
coefficients comprises transforming said set of adjusted intermediate coefficients into a second 
set of intermediate coefficients in said different intermediate space and comparing intermediate 
coefficients of said first set with intermediate coefficients of said second set to determine if 

10 said set of adjusted coefficients is consistent with said first set of intermediate coefficients. 

38. A method according to claim 37 wherein comparing coefficients comprises determining 
u how close the value of at least one intermediate coefficient of said first set is to the value of at 

least one intermediate coefficient of said second set. 

15 

!= 39. A method according to claim 38 wherein determining how close the value of at least 

f- one intermediate coefficient of said first set is to the value of at least one intermediate 

coefficient of said second set comprises using a metric. 

20 40. A method according to any of claims 36 - 39 wherein if said set of adjusted 
intermediate coefficients is not consistent with said first set of intermediate coefficients, the 
value at least one adjusted intermediate coefficient of said set of adjusted intermediate 
coefficients is readjusted to generate a readjusted set of intermediate coefficients so that said 
set of readjusted intermediate coefficients is consistent with said first set of intermediate 

25 coefficients. 

41. A method for recovering an image defined as a function of image coordinates in an 
image space from compressed data that is a function of transform coordinates in a transform 
space the method comprising: 
30 a) using said data to generate a first set of recovered transform coefficients in said 

transform space; 

b) determrning an adjustment range for at least one recovered transform coefficient; 
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c) replacing said at least one recovered transform coefficient by a value in said 
adjustment range responsive to recovered transform coefficients in a neighborhood comprising 
less than all of said recovered transform coefficients of said at least one recovered transform 
coefficient, to generate an adjusted set of recovered transform coefficients; and 
5 d) recovering said image by transforming said adjusted set of recovered transform 

coefficients to said image space. 

42. A method according to claim 41 wherein said neighborhood comprises a plurality of 
recovered transform coefficients including said at least one recovered coefficient, for which 

10 any two recovered coefficients have same values for all the same coordinates except one. 

43. A method for recovering an image defined in an image space from compressed data that 
comprises quantized values and a quantizer for each quantized value that are functions of 
transform coordinates in a transform space the method comprising: 

15 a) determining a set of recovered transform coefficients in which each recovered 

transform coefficient is a product of a quantized value with its quantizer; 

b) determining an adjustment range having an upper and lower limit for at least one 
recovered transform coefficient of said first set of recovered coefficients, wherein the 
difference between said upper and lower limit is less than the magnitude of said quantizer used 

20 to determine said at least one recovered coefficient ; 

c) replacing said at least one recovered transform coefficient by a value in said 
adjustment range to generate an adjusted set of recovered transform coefficients; and 

d) recovering said image by transforming said adjusted set of recovered transform 
coefficients to said image space. 

25 

44. A method for recovering an image defined in an image space from a first set of 
compressed data that comprises quantized values and a quantizer for each quantized value that 
are functions of transform coordinates in a transform space the method comprising: 

a) deterrrmxing a first set of recovered transform coefficients in which each recovered 
3 0 transform coefficient is a product of a quantized value with its quantizer; 

b) detenriming an adjustment range for at least one recovered coefficient; 
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c) replacing said at least one transform coefficient with a value in said adjustment range 
responsive to recovered transform coefficients generated using a second set of related 
compressed data to generate an adjusted set of recovered transform coefficients; and 

d) recovering said image by transforming said adjusted set of recovered transform 
5 coefficients to said image space. 

45. A method according to claim 44 wherein said first and second sets of compressed data 
are generated from different color component images of a color image. 

10 46. A method for adjusting an image having a plurality of image values defined in an image 
space, said image values being generated from compressed data comprising quantized values 
and a quantizer for each quantized value that are functions of transform coordinates in a 
transform space the method comprising: 

determining an adjustment range for at least one image value of said plurality of image 
1 5 values responsive to an average of a plurality said quantizers; 
deterrnining a value in said adjustment range; 

adjusting said image by replacing said at least one image value with said deterniined 

value. 

20 47 A method for adjusting an image according to claim 46 wherein determining an 
adjustment range comprises determining an adjustment range responsive to a fraction less than 
one of me average quantizer. 

48 . A method for adjusting an image having a plurality of image values defined in an image 
25 space, said image values being generated from compressed data comprising quantized values 
and a quantizer for each quantized value that are functions of transform coordinates in a 
transform space the method comprising: 

deterniining an adjustment range for at least one image value of said plurality of image 
values responsive to a maximum or a niinimum quantizer of a plurality of said quantizers; 
30 detemifning a value in said adjustment range; 

adjusting said image by replacing said at least one image value with said determined 

value. 
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49. A method for adjusting an image according to claim 48 wherein determining an 
adjustment range comprises determining an adjustment range responsive to a fraction less than 
one of the maximum or minimum quantizer. 

5 50. A method according to any of claims 46 - 49 wherein said plurality of said quantizers 
comprises all quantizers in said compressed data. 

51. A method according to any of claims 46 - 50 wherein determining a value comprises 
determining a value' responsive to image values of a different related image. 

i 10 

~ 52. A method according to claim 5 1 wherein said image and said different related image are 

color component images of a multicolor color image. 

53. A method according to any of the preceding claims comprising smoothing said 
15 recovered image. 

54. A method according to any of the preceding claims wherein said image is an image 
defined in a three dimensional image space. 

20 55. A method according to any of the preceding claims wherein said image is an image 
defined in a two dimensional space. 

56. A method according to any of the preceding claims wherein said compressed data is 
data generated using a unitary separable transform to transform an image in said image space 

25 into a set of values in said transform space. 

57. A method according to any of the preceding claims wherein said compressed data is 
generated using a JPEG compression method. 

30 58. A method according to any of claims 1- 56 wherein said compressed data is generated 
using an MPEG compression method. 
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59 A method according to any of claims 1- 56 wherein said compressed data is generated 
using a PX64 compression method. 

60. A method according to any of claims 1- 56 wherein said compressed data is generated 
5 using an H261 compression method. 

61. A method according to any of claims 1-56 wherein said compressed data is generated 
using an H263 compression method. 

10 62. A method according to any of claims 1-56 wherein said compressed data is generated 
using an H323 compression method. 

63. A method according to any of claims 1- 56 wherein said compressed data is generated 
using an HDTV compression method. 
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^ Kl was filed on January 1 4,1999 as United States Application No. or PCT International 

Application Number pct/i l99/»qo25 

and was amended SO as in the a an eves to the International Preliminary Examination Report 



I hereby state that I have reviewed and understand the contents of the above identified specification, 
including the claims, as amended by any amendment referred to above. 

i acknowledge the duty to disclose to the United States Patent and Trademark Office ail information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) or 
Section 365(b) of any foreign appllcation(s) for patent or inventor's certificate, or Section 355(a) of any 
PCT International application which designated at least one country other than the United States, 
listed below and have also identified below, by checking the box, any foreign application for patent or 
inventor's certificate or PCT International application having a filing date before that of the application 
on which priority is claimed. 

Prior Foreign Application(s) Priority Not Claimed 



(if applicable) 



□ 



(Number) 



(Country) 



(Day/Month/Year Filed) 



U 



(Number) 



(Country) 



(Day/Month/Year Filed) 



□ 



(Number) 



(Country) 



(Day/Month/Year Filed) 



Fc 
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i hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 
application(s) listed below: 



(Application Serial No.) 



January 1 3^1 999 
(Filing Date} 



(Application Serial No.} 



(Filing Date) 



(Application Serial No.) 



(Filing Date) 



I hereby claim the benefit under 35 U. S. C. Section 120 of any United States application(s), or 
Section 365(0) of any PCT International application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States or PCT International application in the manner provided by the first paragraph of 35 
U.S.C. Section 112, I acknowledge the duty to disclose to the United States Patent and Trademark 
Office all information known to me to be material to patentability as defined in Title 37, C. F. R., 
Section 1.56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



(Application Serial No.) 



(Filing Date) 



(Status) 

(patented, pending, abandoned) 



(Application Serial No.) 



(Filing Date) 



(Status) 

(patented, pending, abandoned) 



(Application Serial No.) 



(Filing Date) 



(Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 
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POWER OF ATTORNEY: As a named inventor, i hereby appoint the following attorney(s) and/or 
ad/ent(s) to prosecute this application and transact all business in the Patent and Trademark Office 
connected therewith, (list name and registration number) 
Paul Fenster, Reg. No. 33,8JX,,_ 
Maier Fenster, Reg. No. .4JU»A6_„ 
Phillip A. Weiss, Reg. No. 19^863— 
Yaakov Schatz, Reg. No. 
'\WilIiam H. Dippert Reg. No. 26^23.-. 
Michael 1. Wolfson, Reg. No. 24J5J0U 
R. Lewis Gable, Reg. No. 22,479 



Send Correspondence to: ^^T 1 H m VSSIh^ 

C owan, Liebowitz & Latman, P.C. 
1 1 33 A venue of ths _Amerjcas 
New YojriyVYJilin6«6?99 

Direct Telephone Calls to: (name and telephone number} 
William H. Dippert (212) 79C-9200 




U ^ 



Full name of second inventor, if any 
MOHADANNY 



Second inventor's signature 



Residence < 
6 Yarmukh Street, Kiryat-U no 55231 , Israel )' 



Post Office Address 

6 Yarmukh Street, Kiryat-Uno 55231, Israel 
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POWER OF ATTORNEY: As a named inventor, \ hereby appoint the following attorney(s) and/or 

agent(s) to prosecute this application and transact all business in the Patent and Trademark Office 

connected therewith, (list name and registration number) 

Paul Fenster, Reg. No. 33,877 

Maier Fenster, Reg. No, 41,016 

Phillip A. Weiss, Reg. No. 19,863 

Yaakov Schatz, Reg. No, 44,320 

William H. Dippert Reg. No. 26,723 

Micnaei l- woiison, Jtveg> no. l 1 *, i jV 

R. Lewis Gable, Reg. No. 22,479 


Send Correspondence to: WilIiam *• Di PP ert > £s * 

Cowan, Liebowitz & Latman, P.C. 
1133 Avenue of the Americas 
New York, NY 10036-6799 




Direct Telephone Calls to: (name and telephone number) 
William H.DiDDert (212)790-9200 


"~ i 1 Ful( name of sole or first inventor 

1 Reuven WEINTRAUB 
s,5 J Sole or first inventor's signature 




1 


Residence 






Citizenship 

Israel _ ._ 






Post Office Address 


















Full name of second inventor, if any 

ido HADANNY 






Second invOftor^^ggrtCf®^ ^ j r J^0Qj^ 
Residence 

6 Yarmukh Street, Kiryst-Uao $5231, Israel _ 






Citizenship 

Post Office Address 

6 Yarmukh Street, Kiryat-Uno 55231, Israel .... 
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